Sep 09 12:17:42.214167 ip-10-0-140-93 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 12:17:42.214175 ip-10-0-140-93 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 12:17:42.214185 ip-10-0-140-93 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 12:17:42.214409 ip-10-0-140-93 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 12:17:52.278232 ip-10-0-140-93 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 12:17:52.278246 ip-10-0-140-93 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b502a19073b04eb9a0472288114a3adc --
Sep 09 12:19:43.571967 ip-10-0-140-93 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 12:19:43.916603 ip-10-0-140-93 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:19:43.916603 ip-10-0-140-93 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 12:19:43.916603 ip-10-0-140-93 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:19:43.916603 ip-10-0-140-93 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 12:19:43.916603 ip-10-0-140-93 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:19:43.917275 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.916685    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 12:19:43.919704 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919688    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:43.919704 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919702    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919707    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919711    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919714    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919717    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919721    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919725    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919730    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919733    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919736    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919739    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919742    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919745    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919749    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919752    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919755    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919757    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919760    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919763    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:43.919779 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919766    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919768    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919776    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919778    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919782    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919785    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919788    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919791    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919794    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919798    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919800    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919803    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919805    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919808    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919811    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919814    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919816    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919819    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919821    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919824    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:43.920225 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919826    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919829    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919831    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919834    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919836    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919839    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919842    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919844    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919847    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919851    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919854    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919857    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919859    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919862    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919866    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919869    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919873    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919876    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919879    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:43.920736 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919882    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919884    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919887    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919889    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919892    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919895    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919897    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919900    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919903    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919905    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919908    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919910    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919913    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919916    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919918    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919921    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919923    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919926    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919929    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:43.921202 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919931    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919933    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919936    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919938    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919941    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919943    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919946    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919948    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919952    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919955    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919957    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919960    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919962    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919965    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919967    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919970    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919972    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919975    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919978    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919980    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.919983    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:43.921730 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920701    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920709    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920714    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920718    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920721    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920724    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920727    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920731    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920733    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920736    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920739    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920741    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920744    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920747    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920750    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920753    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920756    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920759    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920761    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:43.922258 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920765    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920768    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920771    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920773    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920776    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920779    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920787    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920790    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920793    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920795    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920798    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920801    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920803    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920806    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920808    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920811    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920813    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920816    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920818    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920822    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:43.922827 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920825    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920828    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920830    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920833    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920835    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920838    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920840    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920843    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920845    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920848    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920851    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920854    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920857    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920860    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920863    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920865    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920868    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920870    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920873    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920876    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:43.923325 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920878    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920881    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920884    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920886    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920889    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920891    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920894    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920896    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920899    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920902    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920904    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920907    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920910    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920912    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920915    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920918    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920923    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920927    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920930    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:43.923824 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920933    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920936    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920938    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920942    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920944    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920948    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920951    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920953    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920956    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920958    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920961    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920964    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920968    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920971    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920974    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920977    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920980    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920983    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920985    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920988    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:43.924282 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.920991    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921368    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921372    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921376    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921378    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921387    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921389    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921392    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921395    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921398    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921400    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921403    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921406    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921408    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921411    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921414    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921417    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921420    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921422    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921425    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:43.924802 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921428    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921431    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921433    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921436    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921439    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921441    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921444    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921446    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921449    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921452    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921456    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921459    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921462    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921465    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921468    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921470    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921473    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921475    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921478    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:43.925283 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921481    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921483    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921486    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921489    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921492    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921494    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921497    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921500    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921503    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921506    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921509    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921511    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921514    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921517    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921519    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921522    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921526    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921529    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921531    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921534    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:43.925775 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921537    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921539    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921542    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921544    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921547    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921550    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921552    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921555    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921557    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921560    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921562    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921565    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921571    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921573    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921576    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921578    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921581    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921584    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921586    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921588    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:43.926298 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921591    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921595    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921606    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921610    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921614    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921618    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921621    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921623    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921626    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921628    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921631    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921648    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921650    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921653    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921656    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921659    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921661    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921664    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921666    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921669    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:43.926842 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.921671    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921745    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921779    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921786    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921793    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921797    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921801    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921805    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921809    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921813    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921816    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921819    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921822    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921826    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921829    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921832    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921835    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921838    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921841    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921845    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921848    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921851    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921854    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921857    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 12:19:43.927327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921861    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921865    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921868    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921871    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921874    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921877    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921881    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921884    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921887    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921892    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921895    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921898    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921902    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921905    2579 flags.go:64] FLAG: --enable-server="true"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921908    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921913    2579 flags.go:64] FLAG: --event-burst="100"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921917    2579 flags.go:64] FLAG: --event-qps="50"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921920    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921923    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921925    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921929    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921934    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921937    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921940    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921943    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 12:19:43.927913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921947    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921950    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921953    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921956    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921959    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921962    2579 flags.go:64] FLAG: --feature-gates=""
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921966    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921969    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921972    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921975    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921978    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921981    2579 flags.go:64] FLAG: --help="false"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921984    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-140-93.ec2.internal"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921987    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921991    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921993    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.921997    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922000    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922003    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922007    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922010    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922013    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922016    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922020    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 12:19:43.928525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922023    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922026    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922029    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922032    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922036    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922039    2579 flags.go:64] FLAG: --lock-file=""
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922042    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922045    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922048    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922053    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922056    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922059    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922062    2579 flags.go:64] FLAG: --logging-format="text"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922065    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922068    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922072    2579 flags.go:64] FLAG: --manifest-url=""
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922074    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922079    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922082    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922086    2579 flags.go:64] FLAG: --max-pods="110"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922089    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922092    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922095    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922098    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922101    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 12:19:43.929154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922104    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922107    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922115    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922118    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922121    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922124    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922127    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922132    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922135    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922138    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922142    2579 flags.go:64] FLAG: --port="10250"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922146    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922149    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-089a68b22370f4abe"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922152    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922156    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922159    2579 flags.go:64] FLAG: --register-node="true"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922162    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922166    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922168    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922172    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922175    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922178    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922181    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922184    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922187    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 12:19:43.929843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922190    2579 flags.go:64] FLAG: --runonce="false"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922193    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922196    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922199    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922202    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922205    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922208    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922211    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922214    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922217    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922221    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922224    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922227    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922230    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922234    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922237    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922242    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922245    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922249    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922252    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922255    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922259    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922262    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922265    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922268    2579 flags.go:64] FLAG: --v="2"
Sep 09 12:19:43.930461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922272    2579 flags.go:64] FLAG: --version="false"
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922276    2579 flags.go:64] FLAG: --vmodule=""
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922280    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.922283    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922401    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922405    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922408    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922411    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922414    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922417    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922420    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922422    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922425    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922428    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922431    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922434    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922437    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922440    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922443    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922445    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:43.931154 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922448    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922451    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922454    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922456    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922459    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922463    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922466    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922468    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922471    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922474    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922476    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922479    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922481    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922484    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922486    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922490    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922492    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922495    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922497    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922500    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:43.931648 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922503    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922505    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922508    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922510    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922513    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922515    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922518    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922520    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922523    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922526    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922529    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922532    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922535    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922538    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922540    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922543    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922546    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922550    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922553    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922556    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:43.932150 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922558    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922561    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922564    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922566    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922569    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922572    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922574    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922577    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922580    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922582    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922586    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922588    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922591    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922593    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922596    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922599    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922603    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922607    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922610    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922612    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:43.932646 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922615    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922618    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922621    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922624    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922627    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922630    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922648    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922652    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922657    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922661    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922664    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922666    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922669    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922672    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922674    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922677    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922680    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922682    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922686    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:43.933151 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922690    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922693    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922696    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.922698    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.923123    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.930736    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.930751    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.932096    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.932103    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932161    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932168    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932173    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932176    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932180    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:43.933687 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932183    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932187    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932190    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932194    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932197    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932199    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932202    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932206    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932209    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932211    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932214    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932217    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932233    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932238    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932241    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932244    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932246    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932249    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932252    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932254    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:43.934043 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932257    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932260    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932263    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932266    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932269    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932271    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932274    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932277    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932280    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932283    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932285    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932289    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932292    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932294    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932297    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932299    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932302    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932305    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932308    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932310    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:43.934529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932314    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932316    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932320    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932322    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932325    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932327    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932330    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932332    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932337    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932340    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932343    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932346    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932349    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932352    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932355    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932358    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932360    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932363    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932366    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932369    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:43.935105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932371    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932374    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932376    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932379    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932382    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932385    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932388    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932392    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932395    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932397    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932400    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932402    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932405    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932408    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932411    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932414    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932416    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932419    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932421    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:43.935658 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932424    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932426    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932429    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932432    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932434    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932437    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932440    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932442    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932445    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932448    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932451    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932454    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932456    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932459    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932461    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:43.936181 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.932466    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932575    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932580    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932583    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932586    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932589    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932591    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932594    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932597    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932599    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932602    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932604    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932609    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932612    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932616    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932620    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932623    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932626    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932629    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932646    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:19:43.936553 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932651    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932654    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932657    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932660    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932664    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932667    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932670    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932674    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932676    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932679    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932682    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932685    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932687    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932690    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932693    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932695    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932698    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932701    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932703    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932706    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:19:43.937052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932708    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932711    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932714    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932716    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932719    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932722    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932725    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932729    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932732    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932735    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932738    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932740    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932743    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932746    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932748    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932751    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932753    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932756    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932759    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:19:43.937547 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932762    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932764    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932767    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932770    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932772    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932775    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932777    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932780    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932782    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932785    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932788    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932790    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932793    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932796    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932798    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932801    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932803    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932806    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932808    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932811    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:19:43.938054 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932814    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932816    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932819    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932821    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932824    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932827    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932829    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932832    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932834    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932837    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932839    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932842    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932845    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932847    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932850    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932852    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932855    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932857    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932861    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:19:43.938554 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932864    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:19:43.939090 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:43.932866    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:19:43.939090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.932871    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:19:43.939090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.932975    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 12:19:43.939090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.935079    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 12:19:43.939090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.935858    2579 server.go:1019] "Starting client certificate rotation"
Sep 09 12:19:43.939090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.935999    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:19:43.939090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.936658    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:19:43.939090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.938907    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:19:43.940439 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.940412    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:19:43.950275 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.950255    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 09 12:19:43.953110 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.953095    2579 log.go:25] "Validated CRI v1 image API"
Sep 09 12:19:43.953984 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.953962    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 12:19:43.957506 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.957488    2579 fs.go:135] Filesystem UUIDs: map[77e8cdae-64e1-4f06-9ba0-43e578dab6ed:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a56b11d2-f341-432e-bef5-67f515c0ec0f:/dev/nvme0n1p4]
Sep 09 12:19:43.957568 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.957506    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 12:19:43.963355 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.963241    2579 manager.go:216] Machine: {Timestamp:2026-09-09 12:19:43.962024209 +0000 UTC m=+0.286244731 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115524 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21bb87f9dd9adb8f39922b9f09d97a SystemUUID:ec21bb87-f9dd-9adb-8f39-922b9f09d97a BootID:b502a190-73b0-4eb9-a047-2288114a3adc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:61:7d:ac:3b:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:61:7d:ac:3b:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:93:2d:7d:ed:38 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 12:19:43.963355 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.963348    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 12:19:43.963470 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.963433    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 12:19:43.964575 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.964555    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:19:43.965990 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.965960    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 12:19:43.966147 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.965995    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-93.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 12:19:43.967199 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.967189    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 12:19:43.967232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.967201    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 12:19:43.967232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.967209    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:19:43.967841 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.967832    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:19:43.969093 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.969083    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 12:19:43.972611 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.972596    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:19:43.972754 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.972744    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 12:19:43.972792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.972778    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 12:19:43.972823 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.972791    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 12:19:43.972823 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.972814    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 12:19:43.972879 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.972831    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 09 12:19:43.972879 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.972842    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 12:19:43.973307 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.973295    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:19:43.973343 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.973318    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:19:43.975250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.975232    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 12:19:43.977854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.977839    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 12:19:43.977931 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.977869    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 12:19:43.978603 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978592    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 12:19:43.978656 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978609    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 12:19:43.978656 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978616    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 12:19:43.978656 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978621    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 12:19:43.978656 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978629    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 12:19:43.978656 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978649    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 12:19:43.978792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978667    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 12:19:43.978792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978673    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 12:19:43.978792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978679    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 12:19:43.978792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978685    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 12:19:43.978792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978695    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 12:19:43.978792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.978706    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 12:19:43.979455 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.979446    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 12:19:43.979455 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.979455    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 12:19:43.981536 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.981517    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 12:19:43.981644 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:43.981612    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 12:19:43.981758 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:43.981737    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 12:19:43.983448 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.983436    2579 server.go:1268] "Started kubelet"
Sep 09 12:19:43.983770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.983537    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 12:19:43.983770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.983648    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 12:19:43.983881 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.983676    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 12:19:43.987359 ip-10-0-140-93 systemd[1]: Started Kubernetes Kubelet.
Sep 09 12:19:43.988559 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.988356    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 12:19:43.990245 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:43.990141    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 12:19:43.991168 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.991148    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 12:19:43.992717 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.992699    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-46mlf"
Sep 09 12:19:43.994883 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:43.993946    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-93.ec2.internal.18d3a65a59ff9120  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-93.ec2.internal,UID:ip-10-0-140-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-93.ec2.internal,},FirstTimestamp:2026-09-09 12:19:43.983407392 +0000 UTC m=+0.307627913,LastTimestamp:2026-09-09 12:19:43.983407392 +0000 UTC m=+0.307627913,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-93.ec2.internal,}"
Sep 09 12:19:43.996678 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.996662    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 12:19:43.997304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.997285    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 12:19:43.997488 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:43.997469    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:43.997961 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.997942    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 12:19:43.997961 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.997950    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 12:19:43.998089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.997972    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 12:19:43.998089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998051    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 12:19:43.998089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998062    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 12:19:43.998209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998115    2579 factory.go:55] Registering systemd factory
Sep 09 12:19:43.998209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998128    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 09 12:19:43.998397 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998381    2579 factory.go:153] Registering CRI-O factory
Sep 09 12:19:43.998397 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998399    2579 factory.go:223] Registration of the crio container factory successfully
Sep 09 12:19:43.998513 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998459    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 12:19:43.998513 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998478    2579 factory.go:103] Registering Raw factory
Sep 09 12:19:43.998513 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.998494    2579 manager.go:1196] Started watching for new ooms in manager
Sep 09 12:19:43.999283 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:43.999270    2579 manager.go:349] Starting recovery of all containers
Sep 09 12:19:44.001372 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.001348    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-46mlf"
Sep 09 12:19:44.008296 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.008272    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:44.010775 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.010759    2579 manager.go:354] Recovery completed
Sep 09 12:19:44.011953 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.011933    2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-93.ec2.internal\" not found" node="ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.015044 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.015032    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:44.018152 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.018135    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:44.018228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.018172    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:44.018228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.018188    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:44.018791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.018775    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:44.018869 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.018778    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 12:19:44.018869 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.018837    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 12:19:44.018963 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.018872    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:19:44.019560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.019543    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:44.019617 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.019570    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:44.019617 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.019580    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:44.019900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.019888    2579 policy_none.go:49] "None policy: Start"
Sep 09 12:19:44.019930 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.019906    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 12:19:44.019930 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.019919    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 12:19:44.020736 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.020725    2579 policy_none.go:47] "Start"
Sep 09 12:19:44.047383 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.047232    2579 manager.go:337] "Starting Device Plugin manager"
Sep 09 12:19:44.047503 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.047415    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 12:19:44.047503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.047428    2579 server.go:87] "Starting device plugin registration server"
Sep 09 12:19:44.047723 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.047710    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 12:19:44.047777 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.047724    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 12:19:44.047887 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.047872    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 12:19:44.047962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.047951    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 12:19:44.048003 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.047962    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 12:19:44.048545 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.048526    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 12:19:44.048644 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.048578    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.095012 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.094981    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 12:19:44.096188 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.096163    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 12:19:44.096290 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.096211    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 12:19:44.096290 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.096234    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 12:19:44.096405 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.096339    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 12:19:44.098766 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.098746    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:44.148815 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.148779    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:44.149971 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.149954    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:44.150070 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.149992    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:44.150070 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.150007    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:44.150070 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.150035    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.158796 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.158779    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.158862 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.158803    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-93.ec2.internal\": node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.182234 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.182204    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.197227 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.197201    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal"]
Sep 09 12:19:44.197323 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.197302    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:44.199120 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.199105    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:44.199193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.199133    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:44.199193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.199143    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:44.200166 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.200153    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:44.200300 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.200286    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.200359 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.200323    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:44.201759 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.201744    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:44.201834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.201745    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:44.201834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.201797    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:44.201834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.201812    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:44.201834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.201770    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:44.201949 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.201843    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:44.202899 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.202886    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.202948 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.202911    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:19:44.203647 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.203612    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:19:44.203706 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.203663    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:19:44.203706 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.203676    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:19:44.227473 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.227449    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-93.ec2.internal\" not found" node="ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.231845 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.231828    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-93.ec2.internal\" not found" node="ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.282971 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.282944    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.299092 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.299068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cd5d60a79f0f54eda1257e7be2c610e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal\" (UID: \"7cd5d60a79f0f54eda1257e7be2c610e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.299174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.299098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f4f180e5ca224fd9a69355ef754b0028-config\") pod \"kube-apiserver-proxy-ip-10-0-140-93.ec2.internal\" (UID: \"f4f180e5ca224fd9a69355ef754b0028\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.299174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.299115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7cd5d60a79f0f54eda1257e7be2c610e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal\" (UID: \"7cd5d60a79f0f54eda1257e7be2c610e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.383625 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.383591    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.399941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.399898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7cd5d60a79f0f54eda1257e7be2c610e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal\" (UID: \"7cd5d60a79f0f54eda1257e7be2c610e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.400069 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.399958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cd5d60a79f0f54eda1257e7be2c610e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal\" (UID: \"7cd5d60a79f0f54eda1257e7be2c610e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.400069 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.399985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f4f180e5ca224fd9a69355ef754b0028-config\") pod \"kube-apiserver-proxy-ip-10-0-140-93.ec2.internal\" (UID: \"f4f180e5ca224fd9a69355ef754b0028\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.400069 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.400005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7cd5d60a79f0f54eda1257e7be2c610e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal\" (UID: \"7cd5d60a79f0f54eda1257e7be2c610e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.400069 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.400032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f4f180e5ca224fd9a69355ef754b0028-config\") pod \"kube-apiserver-proxy-ip-10-0-140-93.ec2.internal\" (UID: \"f4f180e5ca224fd9a69355ef754b0028\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.400199 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.400073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cd5d60a79f0f54eda1257e7be2c610e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal\" (UID: \"7cd5d60a79f0f54eda1257e7be2c610e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.484342 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.484310    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.530808 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.530774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.535652 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.535611    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:44.584455 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.584418    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.687153 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.687113    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.793773 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.793688    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.901231 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:44.901175    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:44.938187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.938152    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 12:19:44.938756 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.938353    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:19:44.938756 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:44.938354    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:19:45.001981 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:45.001942    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:45.003146 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.003116    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 12:14:44 +0000 UTC" deadline="2028-05-22 23:10:00.117049612 +0000 UTC"
Sep 09 12:19:45.003215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.003147    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14914h50m15.113907058s"
Sep 09 12:19:45.102761 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:45.102685    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:45.146704 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.146678    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:45.203838 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:45.203799    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:45.304891 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:45.304852    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-93.ec2.internal\" not found"
Sep 09 12:19:45.337421 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.337053    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:45.397929 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.397860    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:45.411319 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.411288    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:19:45.412235 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.412219    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal"
Sep 09 12:19:45.422578 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.422552    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:19:45.461635 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:45.461593    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f180e5ca224fd9a69355ef754b0028.slice/crio-bc3280d90ce595ef3eb76cdc409a659a30a9884ccd24019deab551baae7dc915 WatchSource:0}: Error finding container bc3280d90ce595ef3eb76cdc409a659a30a9884ccd24019deab551baae7dc915: Status 404 returned error can't find the container with id bc3280d90ce595ef3eb76cdc409a659a30a9884ccd24019deab551baae7dc915
Sep 09 12:19:45.466035 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.466020    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:19:45.818510 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.818479    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:45.976150 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.976118    2579 apiserver.go:52] "Watching apiserver"
Sep 09 12:19:45.983864 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.983833    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 12:19:45.985403 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.985379    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal","openshift-cluster-node-tuning-operator/tuned-mzgsv","openshift-dns/node-resolver-mnqhx","openshift-multus/multus-additional-cni-plugins-w9xnk","openshift-network-diagnostics/network-check-target-wfgw7","openshift-ovn-kubernetes/ovnkube-node-tngrb","kube-system/global-pull-secret-syncer-d9ktm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t","openshift-image-registry/node-ca-qlcww","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal","openshift-multus/multus-dkqm6","openshift-multus/network-metrics-daemon-mkb58","openshift-network-operator/iptables-alerter-m9lhv","kube-system/konnectivity-agent-cz46c"]
Sep 09 12:19:45.989166 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.989146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:45.989257 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:45.989238    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:19:45.991329 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.991314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:45.993552 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.993527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:45.993552 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.993542    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 12:19:45.993753 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.993627    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cxdms\""
Sep 09 12:19:45.993833 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.993525    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 12:19:45.993948 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.993870    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:19:45.995885 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.995868    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:45.996190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.996048    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tmdcd\""
Sep 09 12:19:45.996285 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.996202    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 12:19:45.996346 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.996314    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 12:19:45.996983 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.996963    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 12:19:45.997933 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.997913    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 12:19:45.998450 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.998428    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 12:19:45.998450 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.998449    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 12:19:45.998657 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.998597    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:45.998763 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:45.998675    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:19:45.998763 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.998730    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bdhdk\""
Sep 09 12:19:45.998882 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.998801    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 12:19:45.998882 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:45.998721    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 12:19:46.000862 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.000833    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.002971 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.002956    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 12:19:46.003195 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.003179    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 12:19:46.003344 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.003326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.003706 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.003681    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 12:19:46.003888 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.003868    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ffcgm\""
Sep 09 12:19:46.003961 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.003878    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 12:19:46.003961 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.003910    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 12:19:46.003961 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.003954    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 12:19:46.005729 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.005705    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:19:46.005936 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.005917    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 12:19:46.005936 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.005930    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q2v6p\""
Sep 09 12:19:46.006096 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.006077    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 12:19:46.006816 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.006787    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 12:19:46.008028 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.008010    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 12:19:46.008115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.008032    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t5bnl\""
Sep 09 12:19:46.008115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.008046    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 12:19:46.008310 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.008294    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.009338 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009319    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/993f53dc-39f2-4111-bd60-ce03f2de7a3d-iptables-alerter-script\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.009444 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d47343aa-225c-4858-810e-414e24704ba5-hosts-file\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.009444 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009373    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d47343aa-225c-4858-810e-414e24704ba5-tmp-dir\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.009444 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009377    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:19:46.009444 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.009444 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009420    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.009698 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009471    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:46.009698 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cni-binary-copy\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.009698 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4x5p\" (UniqueName: \"kubernetes.io/projected/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-kube-api-access-s4x5p\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.009698 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-system-cni-dir\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.009910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009694    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47208fbf-65e4-4cfa-b2bd-7559528bdab4-kubelet-config\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.009910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009726    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrbn4\" (UniqueName: \"kubernetes.io/projected/d47343aa-225c-4858-810e-414e24704ba5-kube-api-access-mrbn4\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.009910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cnibin\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.009910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009784    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.009910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47208fbf-65e4-4cfa-b2bd-7559528bdab4-dbus\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.009910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009837    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.009910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/993f53dc-39f2-4111-bd60-ce03f2de7a3d-host-slash\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.010133 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.009895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gzvd\" (UniqueName: \"kubernetes.io/projected/993f53dc-39f2-4111-bd60-ce03f2de7a3d-kube-api-access-9gzvd\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.010861 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.010783    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.010960 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.010878    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 12:19:46.010960 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.010881    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f4f55\""
Sep 09 12:19:46.010960 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.010909    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:19:46.014482 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.013828    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 12:19:46.014482 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.014186    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 12:19:46.014482 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.014295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.014482 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.014186    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 12:19:46.014806 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.014535    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d9plq\""
Sep 09 12:19:46.016372 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.016350    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 12:19:46.016825 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.016807    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-flrvr\""
Sep 09 12:19:46.016825 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.016821    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:46.016939 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.016886    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:19:46.035171 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.035150    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rbbsc"
Sep 09 12:19:46.045944 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.045921    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rbbsc"
Sep 09 12:19:46.099102 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.099007    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 12:19:46.103065 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.103002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal" event={"ID":"f4f180e5ca224fd9a69355ef754b0028","Type":"ContainerStarted","Data":"bc3280d90ce595ef3eb76cdc409a659a30a9884ccd24019deab551baae7dc915"}
Sep 09 12:19:46.110342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110305    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r88n\" (UniqueName: \"kubernetes.io/projected/14caa40e-0408-4417-b364-72282734fa21-kube-api-access-7r88n\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.110521 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47208fbf-65e4-4cfa-b2bd-7559528bdab4-dbus\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.110521 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gzvd\" (UniqueName: \"kubernetes.io/projected/993f53dc-39f2-4111-bd60-ce03f2de7a3d-kube-api-access-9gzvd\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.110521 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysconfig\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.110521 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110441    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-cni-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.110521 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110466    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-socket-dir-parent\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.110774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-cni-bin\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.110774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110570    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/47208fbf-65e4-4cfa-b2bd-7559528bdab4-dbus\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.110774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-cni-multus\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.110774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-node-log\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.110774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110720    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/993f53dc-39f2-4111-bd60-ce03f2de7a3d-iptables-alerter-script\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.110774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d47343aa-225c-4858-810e-414e24704ba5-hosts-file\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.110774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110762    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d47343aa-225c-4858-810e-414e24704ba5-tmp-dir\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.110774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3a58883-1ecd-4f5b-8949-df19c7f1b4ce-agent-certs\") pod \"konnectivity-agent-cz46c\" (UID: \"f3a58883-1ecd-4f5b-8949-df19c7f1b4ce\") " pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:19:46.111089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110795    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qskfj\" (UniqueName: \"kubernetes.io/projected/f945ef8e-784f-4df0-83c3-b1597dbe942d-kube-api-access-qskfj\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.111089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110926    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d47343aa-225c-4858-810e-414e24704ba5-hosts-file\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.111089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110960    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-systemd\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.111089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.110977    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-log-socket\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.111089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111001    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd947\" (UniqueName: \"kubernetes.io/projected/ea8fac57-2005-432c-9c57-094028153802-kube-api-access-wd947\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.111089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47208fbf-65e4-4cfa-b2bd-7559528bdab4-kubelet-config\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.111089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111050    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbn4\" (UniqueName: \"kubernetes.io/projected/d47343aa-225c-4858-810e-414e24704ba5-kube-api-access-mrbn4\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.111089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-system-cni-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-netns\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d47343aa-225c-4858-810e-414e24704ba5-tmp-dir\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-var-lib-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-cni-bin\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111272    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/47208fbf-65e4-4cfa-b2bd-7559528bdab4-kubelet-config\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111345    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/993f53dc-39f2-4111-bd60-ce03f2de7a3d-iptables-alerter-script\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-registration-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.111391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-cnibin\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111398    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-run-ovn-kubernetes\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111521    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-ovnkube-script-lib\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111550    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f945ef8e-784f-4df0-83c3-b1597dbe942d-host\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-hostroot\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111717    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f945ef8e-784f-4df0-83c3-b1597dbe942d-serviceca\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-run\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-host\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-kubelet\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.111821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-systemd-units\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-systemd\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-cni-netd\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111912    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-ovnkube-config\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111933    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-kubernetes\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.111986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s6dt\" (UniqueName: \"kubernetes.io/projected/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-kube-api-access-2s6dt\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-env-overrides\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cni-binary-copy\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4x5p\" (UniqueName: \"kubernetes.io/projected/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-kube-api-access-s4x5p\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-slash\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-sys-fs\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-cni-binary-copy\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea8fac57-2005-432c-9c57-094028153802-ovn-node-metrics-cert\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3a58883-1ecd-4f5b-8949-df19c7f1b4ce-konnectivity-ca\") pod \"konnectivity-agent-cz46c\" (UID: \"f3a58883-1ecd-4f5b-8949-df19c7f1b4ce\") " pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112300    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/993f53dc-39f2-4111-bd60-ce03f2de7a3d-host-slash\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.112475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-socket-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112387    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/993f53dc-39f2-4111-bd60-ce03f2de7a3d-host-slash\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-device-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysctl-d\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.112447    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-multus-certs\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.112524    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret podName:47208fbf-65e4-4cfa-b2bd-7559528bdab4 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:46.612502152 +0000 UTC m=+2.936722678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret") pod "global-pull-secret-syncer-d9ktm" (UID: "47208fbf-65e4-4cfa-b2bd-7559528bdab4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-etc-kubernetes\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112616    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-etc-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112823    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-sys\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112866    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-var-lib-kubelet\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112923    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-k8s-cni-cncf-io\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.112967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrq2f\" (UniqueName: \"kubernetes.io/projected/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-kube-api-access-nrq2f\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:46.113209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113062    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-run-netns\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-ovn\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysctl-conf\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113235    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14caa40e-0408-4417-b364-72282734fa21-tmp\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-conf-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-daemon-config\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-kubelet\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113370    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p9vg\" (UniqueName: \"kubernetes.io/projected/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-kube-api-access-4p9vg\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-system-cni-dir\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113444    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cnibin\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113502    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-system-cni-dir\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113563    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cnibin\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-modprobe-d\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-lib-modules\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.114048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113673    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/14caa40e-0408-4417-b364-72282734fa21-etc-tuned\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.114623 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.113680    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.114623 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.114070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.114623 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.114251    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-cni-binary-copy\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.121596 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.121565    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:46.121596 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.121599    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:46.121881 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.121614    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mjm2v for pod openshift-network-diagnostics/network-check-target-wfgw7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:46.121881 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.121737    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v podName:2a234ff4-faec-4a2e-8be1-a9952531718c nodeName:}" failed. No retries permitted until 2026-09-09 12:19:46.621715053 +0000 UTC m=+2.945935591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mjm2v" (UniqueName: "kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v") pod "network-check-target-wfgw7" (UID: "2a234ff4-faec-4a2e-8be1-a9952531718c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:46.121881 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.121844    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 12:19:46.126621 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.126559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4x5p\" (UniqueName: \"kubernetes.io/projected/8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae-kube-api-access-s4x5p\") pod \"multus-additional-cni-plugins-w9xnk\" (UID: \"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae\") " pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.126838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.126558    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gzvd\" (UniqueName: \"kubernetes.io/projected/993f53dc-39f2-4111-bd60-ce03f2de7a3d-kube-api-access-9gzvd\") pod \"iptables-alerter-m9lhv\" (UID: \"993f53dc-39f2-4111-bd60-ce03f2de7a3d\") " pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.126838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.126560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrbn4\" (UniqueName: \"kubernetes.io/projected/d47343aa-225c-4858-810e-414e24704ba5-kube-api-access-mrbn4\") pod \"node-resolver-mnqhx\" (UID: \"d47343aa-225c-4858-810e-414e24704ba5\") " pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.163385 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.163353    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd5d60a79f0f54eda1257e7be2c610e.slice/crio-5a71b4c04d993998b2d6960b526aaafe41c199a2973cea9e044d0d08ef4fb12d WatchSource:0}: Error finding container 5a71b4c04d993998b2d6960b526aaafe41c199a2973cea9e044d0d08ef4fb12d: Status 404 returned error can't find the container with id 5a71b4c04d993998b2d6960b526aaafe41c199a2973cea9e044d0d08ef4fb12d
Sep 09 12:19:46.214010 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.213965    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.214207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-system-cni-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.214207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-netns\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.214207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-var-lib-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.214207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-etc-selinux\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.214207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-cni-bin\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-registration-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214234    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-system-cni-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-cnibin\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214292    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-cnibin\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-run-ovn-kubernetes\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-ovnkube-script-lib\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214410    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-cni-bin\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214433    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-registration-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f945ef8e-784f-4df0-83c3-b1597dbe942d-host\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-hostroot\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.214490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f945ef8e-784f-4df0-83c3-b1597dbe942d-serviceca\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-run\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214572    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-run-ovn-kubernetes\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.214589    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-var-lib-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.214650    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs podName:9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:46.714629318 +0000 UTC m=+3.038849852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs") pod "network-metrics-daemon-mkb58" (UID: "9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214699    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-hostroot\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-netns\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214579    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-host\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214750    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-run\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-kubelet\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214795    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-kubelet\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214761    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-host\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f945ef8e-784f-4df0-83c3-b1597dbe942d-host\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.215187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-systemd-units\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-systemd\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-cni-netd\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-ovnkube-config\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-kubernetes\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-systemd\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s6dt\" (UniqueName: \"kubernetes.io/projected/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-kube-api-access-2s6dt\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214945    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-cni-netd\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-env-overrides\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.214981    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-systemd-units\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-slash\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215007    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-kubernetes\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-sys-fs\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-cni-binary-copy\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea8fac57-2005-432c-9c57-094028153802-ovn-node-metrics-cert\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3a58883-1ecd-4f5b-8949-df19c7f1b4ce-konnectivity-ca\") pod \"konnectivity-agent-cz46c\" (UID: \"f3a58883-1ecd-4f5b-8949-df19c7f1b4ce\") " pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215161    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f945ef8e-784f-4df0-83c3-b1597dbe942d-serviceca\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.216167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215165    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-socket-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-device-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysctl-d\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-multus-certs\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-etc-kubernetes\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-sys-fs\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-etc-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-etc-openvswitch\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-sys\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-var-lib-kubelet\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215444    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-ovnkube-script-lib\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215451    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-sys\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-k8s-cni-cncf-io\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrq2f\" (UniqueName: \"kubernetes.io/projected/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-kube-api-access-nrq2f\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215510    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-slash\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-run-netns\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-ovn\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysctl-d\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.217031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysctl-conf\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14caa40e-0408-4417-b364-72282734fa21-tmp\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-conf-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-daemon-config\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-kubelet\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p9vg\" (UniqueName: \"kubernetes.io/projected/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-kube-api-access-4p9vg\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215797    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-modprobe-d\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f3a58883-1ecd-4f5b-8949-df19c7f1b4ce-konnectivity-ca\") pod \"konnectivity-agent-cz46c\" (UID: \"f3a58883-1ecd-4f5b-8949-df19c7f1b4ce\") " pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215821    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-lib-modules\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215292    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-socket-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/14caa40e-0408-4417-b364-72282734fa21-etc-tuned\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-env-overrides\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r88n\" (UniqueName: \"kubernetes.io/projected/14caa40e-0408-4417-b364-72282734fa21-kube-api-access-7r88n\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215942    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-var-lib-kubelet\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysconfig\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-k8s-cni-cncf-io\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215972    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-cni-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-lib-modules\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.215984    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-device-dir\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216009    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-run-multus-certs\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216039    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-cni-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216081    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-kubelet\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216253    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-host-run-netns\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216307    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-run-ovn\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-cni-binary-copy\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-modprobe-d\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-daemon-config\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-etc-kubernetes\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216513    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-socket-dir-parent\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-conf-dir\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216543    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-cni-bin\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-cni-multus\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-node-log\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3a58883-1ecd-4f5b-8949-df19c7f1b4ce-agent-certs\") pod \"konnectivity-agent-cz46c\" (UID: \"f3a58883-1ecd-4f5b-8949-df19c7f1b4ce\") " pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qskfj\" (UniqueName: \"kubernetes.io/projected/f945ef8e-784f-4df0-83c3-b1597dbe942d-kube-api-access-qskfj\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.218900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216675    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-systemd\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216696    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-multus-socket-dir-parent\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-log-socket\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd947\" (UniqueName: \"kubernetes.io/projected/ea8fac57-2005-432c-9c57-094028153802-kube-api-access-wd947\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysconfig\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-systemd\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216879    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-log-socket\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.216622    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-cni-bin\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.217127    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-host-var-lib-cni-multus\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.217172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea8fac57-2005-432c-9c57-094028153802-node-log\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.217374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea8fac57-2005-432c-9c57-094028153802-ovnkube-config\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.217461    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/14caa40e-0408-4417-b364-72282734fa21-etc-sysctl-conf\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.219218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/14caa40e-0408-4417-b364-72282734fa21-etc-tuned\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.219846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.219393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14caa40e-0408-4417-b364-72282734fa21-tmp\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.220877 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.220792    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f3a58883-1ecd-4f5b-8949-df19c7f1b4ce-agent-certs\") pod \"konnectivity-agent-cz46c\" (UID: \"f3a58883-1ecd-4f5b-8949-df19c7f1b4ce\") " pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:19:46.220877 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.220841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea8fac57-2005-432c-9c57-094028153802-ovn-node-metrics-cert\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.227109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.227082    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s6dt\" (UniqueName: \"kubernetes.io/projected/2d7c3c7f-f1aa-4662-8402-a87a3efcae0a-kube-api-access-2s6dt\") pod \"multus-dkqm6\" (UID: \"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a\") " pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.227612 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.227584    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p9vg\" (UniqueName: \"kubernetes.io/projected/3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32-kube-api-access-4p9vg\") pod \"aws-ebs-csi-driver-node-qsq5t\" (UID: \"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.227866 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.227844    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrq2f\" (UniqueName: \"kubernetes.io/projected/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-kube-api-access-nrq2f\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:46.228395 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.228372    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd947\" (UniqueName: \"kubernetes.io/projected/ea8fac57-2005-432c-9c57-094028153802-kube-api-access-wd947\") pod \"ovnkube-node-tngrb\" (UID: \"ea8fac57-2005-432c-9c57-094028153802\") " pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.229913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.229886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r88n\" (UniqueName: \"kubernetes.io/projected/14caa40e-0408-4417-b364-72282734fa21-kube-api-access-7r88n\") pod \"tuned-mzgsv\" (UID: \"14caa40e-0408-4417-b364-72282734fa21\") " pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.230526 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.230498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qskfj\" (UniqueName: \"kubernetes.io/projected/f945ef8e-784f-4df0-83c3-b1597dbe942d-kube-api-access-qskfj\") pod \"node-ca-qlcww\" (UID: \"f945ef8e-784f-4df0-83c3-b1597dbe942d\") " pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.302259 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.302217    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m9lhv"
Sep 09 12:19:46.310158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.310127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mnqhx"
Sep 09 12:19:46.312563 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.312491    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:19:46.315424 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.315395    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993f53dc_39f2_4111_bd60_ce03f2de7a3d.slice/crio-0bf4e545126683e3c93b19e2e12b652bc292598cc143f65b34152820caa475c9 WatchSource:0}: Error finding container 0bf4e545126683e3c93b19e2e12b652bc292598cc143f65b34152820caa475c9: Status 404 returned error can't find the container with id 0bf4e545126683e3c93b19e2e12b652bc292598cc143f65b34152820caa475c9
Sep 09 12:19:46.319341 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.319314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w9xnk"
Sep 09 12:19:46.324085 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.324061    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:19:46.324639 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.324613    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47343aa_225c_4858_810e_414e24704ba5.slice/crio-6b229cd78add3f0cd6ce097cc2e740e6903b512136936012a9d124f26dbb64a6 WatchSource:0}: Error finding container 6b229cd78add3f0cd6ce097cc2e740e6903b512136936012a9d124f26dbb64a6: Status 404 returned error can't find the container with id 6b229cd78add3f0cd6ce097cc2e740e6903b512136936012a9d124f26dbb64a6
Sep 09 12:19:46.330760 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.330692    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qlcww"
Sep 09 12:19:46.336019 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.335995    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:19:46.338384 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.338098    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7d3bc9_ae25_42ce_aa27_2b4f093c1eae.slice/crio-9338a3c52cbf7d82a05a669cd57442e22822a7f81de68b053aaa2a7efc1945ae WatchSource:0}: Error finding container 9338a3c52cbf7d82a05a669cd57442e22822a7f81de68b053aaa2a7efc1945ae: Status 404 returned error can't find the container with id 9338a3c52cbf7d82a05a669cd57442e22822a7f81de68b053aaa2a7efc1945ae
Sep 09 12:19:46.341341 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.341321    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mzgsv"
Sep 09 12:19:46.347132 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.347107    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t"
Sep 09 12:19:46.352144 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.351733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dkqm6"
Sep 09 12:19:46.352144 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.351881    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a58883_1ecd_4f5b_8949_df19c7f1b4ce.slice/crio-1b16364acec6cd95099aed4c4b14745416ee25dd03d33ad84e347dfdd4a4ae43 WatchSource:0}: Error finding container 1b16364acec6cd95099aed4c4b14745416ee25dd03d33ad84e347dfdd4a4ae43: Status 404 returned error can't find the container with id 1b16364acec6cd95099aed4c4b14745416ee25dd03d33ad84e347dfdd4a4ae43
Sep 09 12:19:46.353296 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.353264    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf945ef8e_784f_4df0_83c3_b1597dbe942d.slice/crio-b3ca701b2c76c0d893b214f79b48861f9d84c240e7e4942496dc46145eb77daf WatchSource:0}: Error finding container b3ca701b2c76c0d893b214f79b48861f9d84c240e7e4942496dc46145eb77daf: Status 404 returned error can't find the container with id b3ca701b2c76c0d893b214f79b48861f9d84c240e7e4942496dc46145eb77daf
Sep 09 12:19:46.358052 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.358025    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14caa40e_0408_4417_b364_72282734fa21.slice/crio-64e4cbcbfb030070dcd523fc96e73afcacf673a5a64f8b8f2269fddaff6d9f7b WatchSource:0}: Error finding container 64e4cbcbfb030070dcd523fc96e73afcacf673a5a64f8b8f2269fddaff6d9f7b: Status 404 returned error can't find the container with id 64e4cbcbfb030070dcd523fc96e73afcacf673a5a64f8b8f2269fddaff6d9f7b
Sep 09 12:19:46.361126 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.361100    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3771dfc7_dbc5_4c84_84fc_fa2ab52eaf32.slice/crio-fb6dd73ad61ad1ec51656e308fdaaa082a6c7b7ff3130bc5d14703b70e2c54d6 WatchSource:0}: Error finding container fb6dd73ad61ad1ec51656e308fdaaa082a6c7b7ff3130bc5d14703b70e2c54d6: Status 404 returned error can't find the container with id fb6dd73ad61ad1ec51656e308fdaaa082a6c7b7ff3130bc5d14703b70e2c54d6
Sep 09 12:19:46.370188 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:19:46.370160    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d7c3c7f_f1aa_4662_8402_a87a3efcae0a.slice/crio-2b9a0889147b66a4105559148266323e970f5ad0ecf14660a31a3f9ba117fb1a WatchSource:0}: Error finding container 2b9a0889147b66a4105559148266323e970f5ad0ecf14660a31a3f9ba117fb1a: Status 404 returned error can't find the container with id 2b9a0889147b66a4105559148266323e970f5ad0ecf14660a31a3f9ba117fb1a
Sep 09 12:19:46.620228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.620140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:46.620399 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.620336    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:46.620468 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.620409    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret podName:47208fbf-65e4-4cfa-b2bd-7559528bdab4 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:47.62039146 +0000 UTC m=+3.944611975 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret") pod "global-pull-secret-syncer-d9ktm" (UID: "47208fbf-65e4-4cfa-b2bd-7559528bdab4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:46.721483 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.721441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:46.721673 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:46.721538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:46.721673 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.721615    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:46.721821 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.721704    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs podName:9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:47.721686599 +0000 UTC m=+4.045907136 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs") pod "network-metrics-daemon-mkb58" (UID: "9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:46.721821 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.721709    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:46.721821 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.721730    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:46.721821 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.721744    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mjm2v for pod openshift-network-diagnostics/network-check-target-wfgw7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:46.721821 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:46.721807    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v podName:2a234ff4-faec-4a2e-8be1-a9952531718c nodeName:}" failed. No retries permitted until 2026-09-09 12:19:47.721788583 +0000 UTC m=+4.046009112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mjm2v" (UniqueName: "kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v") pod "network-check-target-wfgw7" (UID: "2a234ff4-faec-4a2e-8be1-a9952531718c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:47.050120 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.050023    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:14:46 +0000 UTC" deadline="2028-02-17 00:11:12.349867313 +0000 UTC"
Sep 09 12:19:47.050120 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.050064    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12611h51m25.299807869s"
Sep 09 12:19:47.116086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.116032    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mzgsv" event={"ID":"14caa40e-0408-4417-b364-72282734fa21","Type":"ContainerStarted","Data":"64e4cbcbfb030070dcd523fc96e73afcacf673a5a64f8b8f2269fddaff6d9f7b"}
Sep 09 12:19:47.117903 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.117867    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qlcww" event={"ID":"f945ef8e-784f-4df0-83c3-b1597dbe942d","Type":"ContainerStarted","Data":"b3ca701b2c76c0d893b214f79b48861f9d84c240e7e4942496dc46145eb77daf"}
Sep 09 12:19:47.119965 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.119939    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mnqhx" event={"ID":"d47343aa-225c-4858-810e-414e24704ba5","Type":"ContainerStarted","Data":"6b229cd78add3f0cd6ce097cc2e740e6903b512136936012a9d124f26dbb64a6"}
Sep 09 12:19:47.121701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.121675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal" event={"ID":"7cd5d60a79f0f54eda1257e7be2c610e","Type":"ContainerStarted","Data":"5a71b4c04d993998b2d6960b526aaafe41c199a2973cea9e044d0d08ef4fb12d"}
Sep 09 12:19:47.123786 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.123761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dkqm6" event={"ID":"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a","Type":"ContainerStarted","Data":"2b9a0889147b66a4105559148266323e970f5ad0ecf14660a31a3f9ba117fb1a"}
Sep 09 12:19:47.127143 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.127116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t" event={"ID":"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32","Type":"ContainerStarted","Data":"fb6dd73ad61ad1ec51656e308fdaaa082a6c7b7ff3130bc5d14703b70e2c54d6"}
Sep 09 12:19:47.128698 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.128673    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cz46c" event={"ID":"f3a58883-1ecd-4f5b-8949-df19c7f1b4ce","Type":"ContainerStarted","Data":"1b16364acec6cd95099aed4c4b14745416ee25dd03d33ad84e347dfdd4a4ae43"}
Sep 09 12:19:47.131064 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.131039    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"e88d423e83ad856a063fd3281e4286f8549296cf884b5f820caf492efa4114b2"}
Sep 09 12:19:47.133589 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.133562    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" event={"ID":"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae","Type":"ContainerStarted","Data":"9338a3c52cbf7d82a05a669cd57442e22822a7f81de68b053aaa2a7efc1945ae"}
Sep 09 12:19:47.135158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.135135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m9lhv" event={"ID":"993f53dc-39f2-4111-bd60-ce03f2de7a3d","Type":"ContainerStarted","Data":"0bf4e545126683e3c93b19e2e12b652bc292598cc143f65b34152820caa475c9"}
Sep 09 12:19:47.629892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.629609    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:47.630122 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:47.630092    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:47.630215 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:47.630168    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret podName:47208fbf-65e4-4cfa-b2bd-7559528bdab4 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:49.630146411 +0000 UTC m=+5.954366925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret") pod "global-pull-secret-syncer-d9ktm" (UID: "47208fbf-65e4-4cfa-b2bd-7559528bdab4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:47.730555 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.730458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:47.730555 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:47.730550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:47.730848 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:47.730714    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:47.730848 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:47.730787    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs podName:9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:49.730764829 +0000 UTC m=+6.054985357 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs") pod "network-metrics-daemon-mkb58" (UID: "9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:47.731279 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:47.731256    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:47.731368 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:47.731289    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:47.731368 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:47.731304    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mjm2v for pod openshift-network-diagnostics/network-check-target-wfgw7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:47.731368 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:47.731356    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v podName:2a234ff4-faec-4a2e-8be1-a9952531718c nodeName:}" failed. No retries permitted until 2026-09-09 12:19:49.731339368 +0000 UTC m=+6.055559894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mjm2v" (UniqueName: "kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v") pod "network-check-target-wfgw7" (UID: "2a234ff4-faec-4a2e-8be1-a9952531718c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:48.050461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:48.050332    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:14:46 +0000 UTC" deadline="2028-02-18 17:29:29.345913799 +0000 UTC"
Sep 09 12:19:48.050461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:48.050397    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12653h9m41.295521239s"
Sep 09 12:19:48.100736 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:48.099868    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:48.100736 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:48.100006    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:19:48.100736 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:48.100410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:48.100736 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:48.100499    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:19:48.100736 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:48.100558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:48.100736 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:48.100645    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:19:48.155166 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:48.155125    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal" event={"ID":"f4f180e5ca224fd9a69355ef754b0028","Type":"ContainerStarted","Data":"71418472a13657471079397a7e4286178d013287b070661adb4b6d9a86d610d5"}
Sep 09 12:19:49.648072 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:49.648029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:49.648738 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:49.648218    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:49.648738 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:49.648288    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret podName:47208fbf-65e4-4cfa-b2bd-7559528bdab4 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:53.648268005 +0000 UTC m=+9.972488522 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret") pod "global-pull-secret-syncer-d9ktm" (UID: "47208fbf-65e4-4cfa-b2bd-7559528bdab4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:49.749799 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:49.749101    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:49.749799 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:49.749150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:49.749799 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:49.749253    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:49.749799 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:49.749305    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs podName:9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6 nodeName:}" failed. No retries permitted until 2026-09-09 12:19:53.749290486 +0000 UTC m=+10.073510997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs") pod "network-metrics-daemon-mkb58" (UID: "9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:49.749799 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:49.749626    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:49.749799 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:49.749640    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:49.749799 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:49.749652    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mjm2v for pod openshift-network-diagnostics/network-check-target-wfgw7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:49.749799 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:49.749762    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v podName:2a234ff4-faec-4a2e-8be1-a9952531718c nodeName:}" failed. No retries permitted until 2026-09-09 12:19:53.74967633 +0000 UTC m=+10.073896842 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mjm2v" (UniqueName: "kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v") pod "network-check-target-wfgw7" (UID: "2a234ff4-faec-4a2e-8be1-a9952531718c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:50.099169 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:50.099135    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:50.099364 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:50.099271    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:19:50.099701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:50.099682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:50.099821 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:50.099799    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:19:50.099889 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:50.099862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:50.099978 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:50.099954    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:19:52.097383 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:52.097344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:52.097853 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:52.097496    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:19:52.097853 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:52.097568    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:52.097853 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:52.097568    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:52.097853 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:52.097736    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:19:52.097853 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:52.097842    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:19:53.682870 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:53.682830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:53.683460 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:53.682994    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:53.683460 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:53.683054    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret podName:47208fbf-65e4-4cfa-b2bd-7559528bdab4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:01.683036314 +0000 UTC m=+18.007256840 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret") pod "global-pull-secret-syncer-d9ktm" (UID: "47208fbf-65e4-4cfa-b2bd-7559528bdab4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:19:53.783517 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:53.783472    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:53.783706 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:53.783552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:53.783706 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:53.783614    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:53.783706 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:53.783672    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:19:53.783706 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:53.783688    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:19:53.783706 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:53.783697    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs podName:9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:01.783677747 +0000 UTC m=+18.107898273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs") pod "network-metrics-daemon-mkb58" (UID: "9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:19:53.783706 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:53.783700    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mjm2v for pod openshift-network-diagnostics/network-check-target-wfgw7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:53.783967 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:53.783736    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v podName:2a234ff4-faec-4a2e-8be1-a9952531718c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:01.783725435 +0000 UTC m=+18.107945957 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mjm2v" (UniqueName: "kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v") pod "network-check-target-wfgw7" (UID: "2a234ff4-faec-4a2e-8be1-a9952531718c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:19:54.097330 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:54.097295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:54.097541 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:54.097449    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:19:54.097880 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:54.097856    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:54.097992 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:54.097960    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:19:54.099035 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:54.099015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:54.099124 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:54.099096    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:19:56.097360 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:56.097286    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:56.097360 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:56.097338    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:56.097910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:56.097286    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:56.097910 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:56.097427    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:19:56.097910 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:56.097547    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:19:56.097910 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:56.097628    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:19:58.096750 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:58.096656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:19:58.096750 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:58.096697    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:19:58.097332 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:19:58.096656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:19:58.097332 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:58.096811    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:19:58.097332 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:58.096949    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:19:58.097332 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:19:58.097026    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:00.096623 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:00.096552    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:00.097078 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:00.096552    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:00.097078 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:00.096710    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:00.097078 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:00.096812    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:00.097078 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:00.096552    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:00.097078 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:00.096918    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:01.743674 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:01.743625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:01.744189 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:01.743779    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:20:01.744189 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:01.743848    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret podName:47208fbf-65e4-4cfa-b2bd-7559528bdab4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:17.743828453 +0000 UTC m=+34.068048975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret") pod "global-pull-secret-syncer-d9ktm" (UID: "47208fbf-65e4-4cfa-b2bd-7559528bdab4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:20:01.844501 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:01.844461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:01.844681 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:01.844524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:01.844681 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:01.844630    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:20:01.844681 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:01.844664    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:20:01.844831 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:01.844689    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:20:01.844831 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:01.844702    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mjm2v for pod openshift-network-diagnostics/network-check-target-wfgw7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:20:01.844831 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:01.844713    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs podName:9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:17.844697134 +0000 UTC m=+34.168917650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs") pod "network-metrics-daemon-mkb58" (UID: "9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:20:01.844831 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:01.844755    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v podName:2a234ff4-faec-4a2e-8be1-a9952531718c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:17.844738127 +0000 UTC m=+34.168958650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mjm2v" (UniqueName: "kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v") pod "network-check-target-wfgw7" (UID: "2a234ff4-faec-4a2e-8be1-a9952531718c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:20:02.096809 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:02.096673    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:02.096809 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:02.096673    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:02.097020 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:02.096814    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:02.097020 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:02.096808    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:02.097020 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:02.096949    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:02.097131 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:02.097023    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:04.097143 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:04.097087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:04.097563 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:04.097202    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:04.097563 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:04.097519    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:04.097662 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:04.097595    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:04.097662 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:04.097656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:04.097752 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:04.097717    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:05.210881 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:05.210573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dkqm6" event={"ID":"2d7c3c7f-f1aa-4662-8402-a87a3efcae0a","Type":"ContainerStarted","Data":"f0eb9763c966164e4c470a21e385e37b13fbdf225145b20f7c6c7b7ae93f77d6"}
Sep 09 12:20:05.213778 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:05.213616    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"5b142abd3dbde7cb1cd3f14924eeda2791100fede4e8dfd9846587f29536a59b"}
Sep 09 12:20:05.216966 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:05.216918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mzgsv" event={"ID":"14caa40e-0408-4417-b364-72282734fa21","Type":"ContainerStarted","Data":"14d99a92ca92642728e88433c6d283cc4011faa84aab147fa3e736309d298c0f"}
Sep 09 12:20:05.232472 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:05.232407    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dkqm6" podStartSLOduration=2.497757946 podStartE2EDuration="21.232384537s" totalImagesPullingTime="18.734626591s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:05.231941627 +0000 UTC m=+21.556162161" watchObservedRunningTime="2026-09-09 12:20:05.232384537 +0000 UTC m=+21.556605071"
Sep 09 12:20:05.232897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:05.232750    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-93.ec2.internal" podStartSLOduration=20.232735459 podStartE2EDuration="20.232735459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:19:48.171785229 +0000 UTC m=+4.496005781" watchObservedRunningTime="2026-09-09 12:20:05.232735459 +0000 UTC m=+21.556955990"
Sep 09 12:20:06.096674 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.096466    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:06.096841 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.096471    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:06.096901 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.096543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:06.096957 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:06.096761    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:06.096957 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:06.096867    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:06.097053 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:06.096995    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:06.220783 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.220746    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t" event={"ID":"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32","Type":"ContainerStarted","Data":"e63431ac90f93716c7d8afbff46b551bb1271a0a0c626dfc22457b9a09971335"}
Sep 09 12:20:06.222304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.222276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cz46c" event={"ID":"f3a58883-1ecd-4f5b-8949-df19c7f1b4ce","Type":"ContainerStarted","Data":"e2fb6af9389a8f1b82ef88d7904a3da68e82d53c37792ef9130b5544d5839df6"}
Sep 09 12:20:06.225058 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.225033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"4cec9411cacf014323c9da90e6367fd1247962dea432d81da047f585ffdf214a"}
Sep 09 12:20:06.225167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.225102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"1298072b1faa6a995e54142bbcdf7051036968e2a52d453e4c3968da52810646"}
Sep 09 12:20:06.225167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.225142    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"ae81d2b5c52d672b8f197750aec4315ec90888683893eff130e0874b48695d3a"}
Sep 09 12:20:06.225264 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.225187    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"2a5a95613aa2ff653a1b9d7251aa4837a0dacc7b598a97ca35ca3e0e274c26c9"}
Sep 09 12:20:06.225264 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.225217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"6eec07326084becf864e9dd256a7963150470de81d02e8adb9c19d355eb1c87f"}
Sep 09 12:20:06.226380 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.226359    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae" containerID="7d77711b93baf21eefd818516114bc5d6de87265728af00d68c95de373f56c98" exitCode=0
Sep 09 12:20:06.226457 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.226440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" event={"ID":"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae","Type":"ContainerDied","Data":"7d77711b93baf21eefd818516114bc5d6de87265728af00d68c95de373f56c98"}
Sep 09 12:20:06.227880 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.227859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qlcww" event={"ID":"f945ef8e-784f-4df0-83c3-b1597dbe942d","Type":"ContainerStarted","Data":"48ed9c664ad1c34e10160a55260deb1596a7884b45825b763c2ba138a8632dd4"}
Sep 09 12:20:06.229292 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.229273    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mnqhx" event={"ID":"d47343aa-225c-4858-810e-414e24704ba5","Type":"ContainerStarted","Data":"9c04492450be0220561b78e577774fb27e0b7610fee64a76e9557a40b9a66895"}
Sep 09 12:20:06.230824 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.230805    2579 generic.go:356] "Generic (PLEG): container finished" podID="7cd5d60a79f0f54eda1257e7be2c610e" containerID="307e1d0ee6a4930060f46933e311601a1f84023132d28ad5705ff8e3e96f1fc4" exitCode=0
Sep 09 12:20:06.230911 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.230885    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal" event={"ID":"7cd5d60a79f0f54eda1257e7be2c610e","Type":"ContainerDied","Data":"307e1d0ee6a4930060f46933e311601a1f84023132d28ad5705ff8e3e96f1fc4"}
Sep 09 12:20:06.237256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.237220    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cz46c" podStartSLOduration=3.589538881 podStartE2EDuration="22.237209666s" totalImagesPullingTime="18.647670785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:06.236894261 +0000 UTC m=+22.561114791" watchObservedRunningTime="2026-09-09 12:20:06.237209666 +0000 UTC m=+22.561430197"
Sep 09 12:20:06.237339 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.237305    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mzgsv" podStartSLOduration=3.596689825 podStartE2EDuration="22.237298276s" totalImagesPullingTime="18.640608451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:05.255353507 +0000 UTC m=+21.579574039" watchObservedRunningTime="2026-09-09 12:20:06.237298276 +0000 UTC m=+22.561518807"
Sep 09 12:20:06.251679 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.251629    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qlcww" podStartSLOduration=3.622584208 podStartE2EDuration="22.251616652s" totalImagesPullingTime="18.629032444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:06.251149068 +0000 UTC m=+22.575369600" watchObservedRunningTime="2026-09-09 12:20:06.251616652 +0000 UTC m=+22.575837183"
Sep 09 12:20:06.279034 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:06.278387    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mnqhx" podStartSLOduration=3.621145241 podStartE2EDuration="22.278370447s" totalImagesPullingTime="18.657225206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:06.278202862 +0000 UTC m=+22.602423394" watchObservedRunningTime="2026-09-09 12:20:06.278370447 +0000 UTC m=+22.602590979"
Sep 09 12:20:07.238492 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:07.237692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m9lhv" event={"ID":"993f53dc-39f2-4111-bd60-ce03f2de7a3d","Type":"ContainerStarted","Data":"576d936a7efc1c9df4d733fb366fdf71c2b0da737ae617084dfacba0681c419f"}
Sep 09 12:20:07.240998 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:07.240944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal" event={"ID":"7cd5d60a79f0f54eda1257e7be2c610e","Type":"ContainerStarted","Data":"346c1501324b349c431474a57dfed2f98832bfb73ce6234280ff2e6d1020bb5e"}
Sep 09 12:20:07.252183 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:07.252117    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m9lhv" podStartSLOduration=4.587263975 podStartE2EDuration="23.252100711s" totalImagesPullingTime="18.664836736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:07.251727236 +0000 UTC m=+23.575947769" watchObservedRunningTime="2026-09-09 12:20:07.252100711 +0000 UTC m=+23.576321243"
Sep 09 12:20:07.266567 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:07.266520    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-93.ec2.internal" podStartSLOduration=22.266503708 podStartE2EDuration="22.266503708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:07.26630726 +0000 UTC m=+23.590527791" watchObservedRunningTime="2026-09-09 12:20:07.266503708 +0000 UTC m=+23.590724241"
Sep 09 12:20:07.993062 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:07.992469    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 12:20:08.065728 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.065601    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T12:20:07.992760038Z","UUID":"47c6ebd8-b4a8-454d-9b74-c1993869dcdc","Handler":null,"Name":"","Endpoint":""}
Sep 09 12:20:08.067309 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.067284    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 12:20:08.067440 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.067319    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 12:20:08.097184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.097151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:08.097184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.097173    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:08.097384 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:08.097278    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:08.097384 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.097324    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:08.097486 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:08.097439    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:08.097560 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:08.097534    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:08.245621 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.245482    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t" event={"ID":"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32","Type":"ContainerStarted","Data":"40ac4288dfc87fb4f5f104bd91e9bb1e127ee36f9ba52c2426be8c4fd9680232"}
Sep 09 12:20:08.249441 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.249411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"859eb74e6bb15a052588449d8977fd8d523ae350a141728e50e797d25fbae6db"}
Sep 09 12:20:08.299224 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.299191    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:20:08.299837 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:08.299811    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:20:09.254317 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:09.254076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t" event={"ID":"3771dfc7-dbc5-4c84-84fc-fa2ab52eaf32","Type":"ContainerStarted","Data":"5ae676d768cb867acb13498a6f58d2ab0ffaa075f3d74e07e0f1f99eb421af18"}
Sep 09 12:20:09.254745 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:09.254351    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:20:09.254819 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:09.254777    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cz46c"
Sep 09 12:20:09.272457 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:09.272408    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qsq5t" podStartSLOduration=3.061886632 podStartE2EDuration="25.27239438s" totalImagesPullingTime="22.210507748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:09.272328642 +0000 UTC m=+25.596549176" watchObservedRunningTime="2026-09-09 12:20:09.27239438 +0000 UTC m=+25.596614929"
Sep 09 12:20:10.100654 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:10.100616    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:10.100829 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:10.100668    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:10.100829 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:10.100676    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:10.100829 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:10.100771    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:10.100966 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:10.100886    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:10.101007 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:10.100982    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:11.262560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:11.262343    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" event={"ID":"ea8fac57-2005-432c-9c57-094028153802","Type":"ContainerStarted","Data":"fc43b71580e5255fe30384ec2baaea79e949eced37a7014a501913088a47cbbe"}
Sep 09 12:20:11.263025 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:11.262587    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:20:11.263025 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:11.262661    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:20:11.264425 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:11.264399    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae" containerID="452e3be66a226ad904b81d484af417bdb26e469337e69805ac79f2a7ab59393d" exitCode=0
Sep 09 12:20:11.264562 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:11.264479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" event={"ID":"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae","Type":"ContainerDied","Data":"452e3be66a226ad904b81d484af417bdb26e469337e69805ac79f2a7ab59393d"}
Sep 09 12:20:11.277946 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:11.277928    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:20:11.291112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:11.291073    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb" podStartSLOduration=8.610105521 podStartE2EDuration="27.29106058s" totalImagesPullingTime="18.680955059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:11.290826417 +0000 UTC m=+27.615046970" watchObservedRunningTime="2026-09-09 12:20:11.29106058 +0000 UTC m=+27.615281111"
Sep 09 12:20:12.097115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.097086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:12.097115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.097108    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:12.097338 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.097096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:12.097338 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:12.097190    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:12.097338 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:12.097245    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:12.097501 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:12.097355    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:12.266667 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.266614    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:20:12.280927 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.280904    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:20:12.789216 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.789181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d9ktm"]
Sep 09 12:20:12.789380 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.789303    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:12.789448 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:12.789425    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:12.790513 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.790466    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wfgw7"]
Sep 09 12:20:12.790680 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.790573    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:12.791016 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.790995    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mkb58"]
Sep 09 12:20:12.791151 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:12.791129    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:12.791274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:12.791188    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:12.791274 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:12.791267    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:13.270812 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:13.270781    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae" containerID="bb17abfc7dc4843e76386a23572292c992bc458de3fe9a8de2ed3885e4c13926" exitCode=0
Sep 09 12:20:13.271652 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:13.270862    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" event={"ID":"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae","Type":"ContainerDied","Data":"bb17abfc7dc4843e76386a23572292c992bc458de3fe9a8de2ed3885e4c13926"}
Sep 09 12:20:14.101242 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:14.100555    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:14.101242 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:14.100959    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:14.101242 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:14.101036    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:14.101242 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:14.101141    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:14.102049 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:14.101334    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:14.102049 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:14.101487    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:15.277665 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:15.277571    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae" containerID="8557bd0a8bd77083ff0c558dc577f622ae04827563028a62165fe9a90da22fd8" exitCode=0
Sep 09 12:20:15.277665 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:15.277613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" event={"ID":"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae","Type":"ContainerDied","Data":"8557bd0a8bd77083ff0c558dc577f622ae04827563028a62165fe9a90da22fd8"}
Sep 09 12:20:16.097423 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:16.097389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:16.097550 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:16.097395    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:16.097550 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:16.097495    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d9ktm" podUID="47208fbf-65e4-4cfa-b2bd-7559528bdab4"
Sep 09 12:20:16.097656 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:16.097395    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:16.097656 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:16.097560    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wfgw7" podUID="2a234ff4-faec-4a2e-8be1-a9952531718c"
Sep 09 12:20:16.097656 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:16.097631    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mkb58" podUID="9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6"
Sep 09 12:20:17.762752 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:17.762711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:17.763226 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:17.762885    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:20:17.763226 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:17.762971    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret podName:47208fbf-65e4-4cfa-b2bd-7559528bdab4 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:49.762948426 +0000 UTC m=+66.087168951 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret") pod "global-pull-secret-syncer-d9ktm" (UID: "47208fbf-65e4-4cfa-b2bd-7559528bdab4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:20:17.863373 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:17.863287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:17.863373 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:17.863343    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:17.863559 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:17.863457    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:20:17.863559 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:17.863471    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:20:17.863559 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:17.863491    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:20:17.863559 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:17.863502    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mjm2v for pod openshift-network-diagnostics/network-check-target-wfgw7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:20:17.863559 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:17.863525    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs podName:9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:49.863506224 +0000 UTC m=+66.187726739 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs") pod "network-metrics-daemon-mkb58" (UID: "9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:20:17.863559 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:17.863546    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v podName:2a234ff4-faec-4a2e-8be1-a9952531718c nodeName:}" failed. No retries permitted until 2026-09-09 12:20:49.863530861 +0000 UTC m=+66.187751382 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mjm2v" (UniqueName: "kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v") pod "network-check-target-wfgw7" (UID: "2a234ff4-faec-4a2e-8be1-a9952531718c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:20:17.976316 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:17.976288    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-93.ec2.internal" event="NodeReady"
Sep 09 12:20:17.976490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:17.976422    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 12:20:18.026447 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.026409    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9vslz"]
Sep 09 12:20:18.102412 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.102375    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.104796 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.104764    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 12:20:18.104952 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.104861    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hsrxf\""
Sep 09 12:20:18.104952 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.104772    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 12:20:18.106310 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.106287    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j46h4"]
Sep 09 12:20:18.132298 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.132239    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j46h4"]
Sep 09 12:20:18.132411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.132328    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:18.132411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.132333    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9vslz"]
Sep 09 12:20:18.132411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.132329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:18.132411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.132337    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j46h4"
Sep 09 12:20:18.132411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.132363    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:18.135722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135348    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 12:20:18.135722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135357    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lblnd\""
Sep 09 12:20:18.135722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135439    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 12:20:18.135722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135439    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:20:18.135722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135583    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:20:18.135722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135594    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:20:18.135722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135599    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wjmxh\""
Sep 09 12:20:18.136137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135913    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 12:20:18.136137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.135980    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:20:18.136137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.136043    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kls2h\""
Sep 09 12:20:18.266775 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.266743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z5b8\" (UniqueName: \"kubernetes.io/projected/949d2d6e-2a7a-4d2b-9d52-9303f3540d23-kube-api-access-8z5b8\") pod \"ingress-canary-j46h4\" (UID: \"949d2d6e-2a7a-4d2b-9d52-9303f3540d23\") " pod="openshift-ingress-canary/ingress-canary-j46h4"
Sep 09 12:20:18.266958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.266794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.266958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.266820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-tmp-dir\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.266958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.266844    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfm5k\" (UniqueName: \"kubernetes.io/projected/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-kube-api-access-dfm5k\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.266958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.266876    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-config-volume\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.266958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.266918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/949d2d6e-2a7a-4d2b-9d52-9303f3540d23-cert\") pod \"ingress-canary-j46h4\" (UID: \"949d2d6e-2a7a-4d2b-9d52-9303f3540d23\") " pod="openshift-ingress-canary/ingress-canary-j46h4"
Sep 09 12:20:18.367547 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.367512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-config-volume\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.367732 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.367556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/949d2d6e-2a7a-4d2b-9d52-9303f3540d23-cert\") pod \"ingress-canary-j46h4\" (UID: \"949d2d6e-2a7a-4d2b-9d52-9303f3540d23\") " pod="openshift-ingress-canary/ingress-canary-j46h4"
Sep 09 12:20:18.367732 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.367657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z5b8\" (UniqueName: \"kubernetes.io/projected/949d2d6e-2a7a-4d2b-9d52-9303f3540d23-kube-api-access-8z5b8\") pod \"ingress-canary-j46h4\" (UID: \"949d2d6e-2a7a-4d2b-9d52-9303f3540d23\") " pod="openshift-ingress-canary/ingress-canary-j46h4"
Sep 09 12:20:18.367732 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.367711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.367900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.367734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-tmp-dir\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.367900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.367761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfm5k\" (UniqueName: \"kubernetes.io/projected/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-kube-api-access-dfm5k\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.367900 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:18.367849    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:18.368035 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:18.367924    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls podName:0e58e54d-f883-4fbb-8ddf-b8f65dc39874 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:18.867900853 +0000 UTC m=+35.192121382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls") pod "dns-default-9vslz" (UID: "0e58e54d-f883-4fbb-8ddf-b8f65dc39874") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:18.368101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.368038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-tmp-dir\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.368152 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.368130    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-config-volume\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.372100 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.372071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/949d2d6e-2a7a-4d2b-9d52-9303f3540d23-cert\") pod \"ingress-canary-j46h4\" (UID: \"949d2d6e-2a7a-4d2b-9d52-9303f3540d23\") " pod="openshift-ingress-canary/ingress-canary-j46h4"
Sep 09 12:20:18.376165 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.376141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfm5k\" (UniqueName: \"kubernetes.io/projected/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-kube-api-access-dfm5k\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.376405 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.376385    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z5b8\" (UniqueName: \"kubernetes.io/projected/949d2d6e-2a7a-4d2b-9d52-9303f3540d23-kube-api-access-8z5b8\") pod \"ingress-canary-j46h4\" (UID: \"949d2d6e-2a7a-4d2b-9d52-9303f3540d23\") " pod="openshift-ingress-canary/ingress-canary-j46h4"
Sep 09 12:20:18.462932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.462894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j46h4"
Sep 09 12:20:18.870677 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:18.870581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:18.871047 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:18.870761    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:18.871047 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:18.870841    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls podName:0e58e54d-f883-4fbb-8ddf-b8f65dc39874 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:19.870818718 +0000 UTC m=+36.195039234 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls") pod "dns-default-9vslz" (UID: "0e58e54d-f883-4fbb-8ddf-b8f65dc39874") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:19.166436 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:19.166341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:19.276858 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:19.276814    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:19.638740 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:19.638706    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:19.879499 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:19.879461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:19.879956 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:19.879588    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:19.879956 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:19.879680    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls podName:0e58e54d-f883-4fbb-8ddf-b8f65dc39874 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:21.8796581 +0000 UTC m=+38.203878623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls") pod "dns-default-9vslz" (UID: "0e58e54d-f883-4fbb-8ddf-b8f65dc39874") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:19.892609 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:19.892075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:19.959748 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:19.959498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:20.182253 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:20.182227    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j46h4"]
Sep 09 12:20:20.185579 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:20.185554    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949d2d6e_2a7a_4d2b_9d52_9303f3540d23.slice/crio-cad609a11f2768cc976a2c373cc93c27b512e0f42d8e790ea4d5976796bdc039 WatchSource:0}: Error finding container cad609a11f2768cc976a2c373cc93c27b512e0f42d8e790ea4d5976796bdc039: Status 404 returned error can't find the container with id cad609a11f2768cc976a2c373cc93c27b512e0f42d8e790ea4d5976796bdc039
Sep 09 12:20:20.289559 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:20.289530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j46h4" event={"ID":"949d2d6e-2a7a-4d2b-9d52-9303f3540d23","Type":"ContainerStarted","Data":"cad609a11f2768cc976a2c373cc93c27b512e0f42d8e790ea4d5976796bdc039"}
Sep 09 12:20:21.298240 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:21.298206    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae" containerID="b9acfccfba381746bb7cb727f45b82a5f96f2b7cab0f0b7d4a7028ac8c2cbf6c" exitCode=0
Sep 09 12:20:21.298920 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:21.298260    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" event={"ID":"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae","Type":"ContainerDied","Data":"b9acfccfba381746bb7cb727f45b82a5f96f2b7cab0f0b7d4a7028ac8c2cbf6c"}
Sep 09 12:20:21.894914 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:21.894878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:21.895065 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:21.895034    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:21.895117 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:21.895096    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls podName:0e58e54d-f883-4fbb-8ddf-b8f65dc39874 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:25.895077811 +0000 UTC m=+42.219298337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls") pod "dns-default-9vslz" (UID: "0e58e54d-f883-4fbb-8ddf-b8f65dc39874") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:22.304280 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:22.304234    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae" containerID="13140c615bd93728b449b1226b81622bf8d9a5d2520c4e47f15b7706ab6a5b54" exitCode=0
Sep 09 12:20:22.304785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:22.304292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" event={"ID":"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae","Type":"ContainerDied","Data":"13140c615bd93728b449b1226b81622bf8d9a5d2520c4e47f15b7706ab6a5b54"}
Sep 09 12:20:23.309852 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:23.309825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" event={"ID":"8a7d3bc9-ae25-42ce-aa27-2b4f093c1eae","Type":"ContainerStarted","Data":"9393dedd682c97048117d3ab3247a5feffc5a43b11f6c7ba52b7798d55b23014"}
Sep 09 12:20:23.334944 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:23.334895    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w9xnk" podStartSLOduration=8.511761632 podStartE2EDuration="39.334876378s" totalImagesPullingTime="30.823114746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:23.333693013 +0000 UTC m=+39.657913546" watchObservedRunningTime="2026-09-09 12:20:23.334876378 +0000 UTC m=+39.659096909"
Sep 09 12:20:23.457221 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:23.457187    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:23.522811 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:23.522695    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:24.313834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:24.313799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j46h4" event={"ID":"949d2d6e-2a7a-4d2b-9d52-9303f3540d23","Type":"ContainerStarted","Data":"4445de116562b54722c63c26cad1c6ed3439604409ad70f7146c817e3555c643"}
Sep 09 12:20:24.329981 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:24.329936    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j46h4" podStartSLOduration=3.247826799 podStartE2EDuration="6.329921862s" totalImagesPullingTime="3.082095063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:24.329154129 +0000 UTC m=+40.653374660" watchObservedRunningTime="2026-09-09 12:20:24.329921862 +0000 UTC m=+40.654142463"
Sep 09 12:20:24.675611 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:24.675524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:24.740219 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:24.740187    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:25.174324 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.174295    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ftk7f"]
Sep 09 12:20:25.179591 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.179556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:25.193533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.193512    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ftk7f"]
Sep 09 12:20:25.193657 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.193600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.195803 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.195780    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 12:20:25.196127 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.196108    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 12:20:25.196361 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.196141    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 12:20:25.196361 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.196179    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tndcj\""
Sep 09 12:20:25.196361 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.196141    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 12:20:25.274099 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.274069    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz"]
Sep 09 12:20:25.279828 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.279800    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:25.299066 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.299045    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz"]
Sep 09 12:20:25.299179 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.299127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz"
Sep 09 12:20:25.301270 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.301253    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 09 12:20:25.301375 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.301253    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 09 12:20:25.301375 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.301344    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zs22j\""
Sep 09 12:20:25.321388 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.321365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.321713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.321397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4f4314f-3c74-415b-8223-05252a5b04db-crio-socket\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.321713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.321413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn7cp\" (UniqueName: \"kubernetes.io/projected/d4f4314f-3c74-415b-8223-05252a5b04db-kube-api-access-cn7cp\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.321713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.321435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.321713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.321507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4f4314f-3c74-415b-8223-05252a5b04db-data-volume\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.341803 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.341780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:25.422881 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.422849    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4f4314f-3c74-415b-8223-05252a5b04db-data-volume\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.423007 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.422902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmfbh\" (UniqueName: \"kubernetes.io/projected/4603b301-6bd9-4618-83c9-af2d40a95383-kube-api-access-dmfbh\") pod \"migrator-fdbb8fdff-q2rvz\" (UID: \"4603b301-6bd9-4618-83c9-af2d40a95383\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz"
Sep 09 12:20:25.423091 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.423078    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.423143 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.423102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4f4314f-3c74-415b-8223-05252a5b04db-crio-socket\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.423143 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.423118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn7cp\" (UniqueName: \"kubernetes.io/projected/d4f4314f-3c74-415b-8223-05252a5b04db-kube-api-access-cn7cp\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.423231 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.423146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.423295 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.423279    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:20:25.423344 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.423285    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:25.423383 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.423333    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4f4314f-3c74-415b-8223-05252a5b04db-data-volume\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.423383 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.423332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4f4314f-3c74-415b-8223-05252a5b04db-crio-socket\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.423474 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.423403    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:25.9233534 +0000 UTC m=+42.247573923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:25.423474 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.423440    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:25.923423698 +0000 UTC m=+42.247644207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : configmap "kube-rbac-proxy" not found
Sep 09 12:20:25.432686 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.432629    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn7cp\" (UniqueName: \"kubernetes.io/projected/d4f4314f-3c74-415b-8223-05252a5b04db-kube-api-access-cn7cp\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.523629 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.523603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmfbh\" (UniqueName: \"kubernetes.io/projected/4603b301-6bd9-4618-83c9-af2d40a95383-kube-api-access-dmfbh\") pod \"migrator-fdbb8fdff-q2rvz\" (UID: \"4603b301-6bd9-4618-83c9-af2d40a95383\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz"
Sep 09 12:20:25.531712 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.531689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmfbh\" (UniqueName: \"kubernetes.io/projected/4603b301-6bd9-4618-83c9-af2d40a95383-kube-api-access-dmfbh\") pod \"migrator-fdbb8fdff-q2rvz\" (UID: \"4603b301-6bd9-4618-83c9-af2d40a95383\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz"
Sep 09 12:20:25.591907 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.591879    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:25.607710 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.607690    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz"
Sep 09 12:20:25.697527 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.697427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:25.770015 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.769978    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:25.891712 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.891665    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz"]
Sep 09 12:20:25.894897 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:25.894866    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4603b301_6bd9_4618_83c9_af2d40a95383.slice/crio-c2e78fad42cbe465fdcd94fcd1a47639b859bb5027644e1d2b833ad701c85285 WatchSource:0}: Error finding container c2e78fad42cbe465fdcd94fcd1a47639b859bb5027644e1d2b833ad701c85285: Status 404 returned error can't find the container with id c2e78fad42cbe465fdcd94fcd1a47639b859bb5027644e1d2b833ad701c85285
Sep 09 12:20:25.927046 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.927016    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.927236 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.927160    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:20:25.927236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.927173    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:25.927236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:25.927200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:25.927400 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.927241    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:26.927217948 +0000 UTC m=+43.251438471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : configmap "kube-rbac-proxy" not found
Sep 09 12:20:25.927400 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.927299    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:25.927400 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.927304    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:25.927400 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.927359    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:26.927345025 +0000 UTC m=+43.251565548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:25.927400 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:25.927378    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls podName:0e58e54d-f883-4fbb-8ddf-b8f65dc39874 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:33.927368688 +0000 UTC m=+50.251589218 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls") pod "dns-default-9vslz" (UID: "0e58e54d-f883-4fbb-8ddf-b8f65dc39874") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:26.320485 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:26.320449    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz" event={"ID":"4603b301-6bd9-4618-83c9-af2d40a95383","Type":"ContainerStarted","Data":"c2e78fad42cbe465fdcd94fcd1a47639b859bb5027644e1d2b833ad701c85285"}
Sep 09 12:20:26.433994 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:26.433961    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:26.498238 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:26.498204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:26.934075 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:26.934041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:26.934231 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:26.934087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:26.934231 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:26.934182    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:20:26.934231 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:26.934202    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:26.934341 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:26.934238    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:28.93422368 +0000 UTC m=+45.258444189 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : configmap "kube-rbac-proxy" not found
Sep 09 12:20:26.934341 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:26.934251    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:28.934245 +0000 UTC m=+45.258465509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:27.139624 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:27.139591    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mnqhx_d47343aa-225c-4858-810e-414e24704ba5/dns-node-resolver/0.log"
Sep 09 12:20:27.995391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:27.995359    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-x5zkm"]
Sep 09 12:20:28.016857 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.016834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-x5zkm"]
Sep 09 12:20:28.016974 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.016923    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.019190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.019169    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 09 12:20:28.019851 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.019828    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-mpl84\""
Sep 09 12:20:28.019967 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.019868    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 09 12:20:28.019967 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.019884    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 09 12:20:28.020145 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.020131    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 09 12:20:28.070911 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.070883    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:28.139337 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.139315    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qlcww_f945ef8e-784f-4df0-83c3-b1597dbe942d/node-ca/0.log"
Sep 09 12:20:28.143600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.143581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-signing-key\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.143689 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.143657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-signing-cabundle\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.143689 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.143680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tf6m\" (UniqueName: \"kubernetes.io/projected/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-kube-api-access-8tf6m\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.244811 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.244786    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-signing-key\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.244905 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.244837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-signing-cabundle\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.244962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.244943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tf6m\" (UniqueName: \"kubernetes.io/projected/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-kube-api-access-8tf6m\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.245377 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.245357    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-signing-cabundle\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.246939 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.246901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-signing-key\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.252907 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.252889    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tf6m\" (UniqueName: \"kubernetes.io/projected/e1241c5f-6380-4ffe-aabd-2e50f0bbbee3-kube-api-access-8tf6m\") pod \"service-ca-75d99c484-x5zkm\" (UID: \"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3\") " pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.325138 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.325114    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-x5zkm"
Sep 09 12:20:28.327205 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.327183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz" event={"ID":"4603b301-6bd9-4618-83c9-af2d40a95383","Type":"ContainerStarted","Data":"ca283612a34e5e0688a084aa54eed559b9e98b4e3e6067a557d824b2e2943e83"}
Sep 09 12:20:28.327329 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.327228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz" event={"ID":"4603b301-6bd9-4618-83c9-af2d40a95383","Type":"ContainerStarted","Data":"7ca2ed04ef9aa9f64ea0f39e9eecca9b84ef6f87d874aeacb4fb3a9543607728"}
Sep 09 12:20:28.343334 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.343295    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-q2rvz" podStartSLOduration=1.186899574 podStartE2EDuration="3.343279984s" totalImagesPullingTime="2.15638041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:28.343021102 +0000 UTC m=+44.667241634" watchObservedRunningTime="2026-09-09 12:20:28.343279984 +0000 UTC m=+44.667500521"
Sep 09 12:20:28.535260 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.535177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:28.606230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.606193    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-x5zkm"]
Sep 09 12:20:28.609103 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.609066    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:28.610881 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:28.610857    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1241c5f_6380_4ffe_aabd_2e50f0bbbee3.slice/crio-44ebf73454d30e4300f0b72f13802be0fd92e77637080f42e6b7100ef01a6c0e WatchSource:0}: Error finding container 44ebf73454d30e4300f0b72f13802be0fd92e77637080f42e6b7100ef01a6c0e: Status 404 returned error can't find the container with id 44ebf73454d30e4300f0b72f13802be0fd92e77637080f42e6b7100ef01a6c0e
Sep 09 12:20:28.828544 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.828464    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:28.939861 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.939829    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j46h4_949d2d6e-2a7a-4d2b-9d52-9303f3540d23/serve-healthcheck-canary/0.log"
Sep 09 12:20:28.951231 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.951207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:28.951364 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:28.951244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:28.951364 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:28.951330    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:20:28.951470 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:28.951364    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:28.951470 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:28.951377    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:32.951365474 +0000 UTC m=+49.275585984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : configmap "kube-rbac-proxy" not found
Sep 09 12:20:28.951470 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:28.951422    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:32.95140488 +0000 UTC m=+49.275625405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:29.331795 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:29.331754    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-x5zkm" event={"ID":"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3","Type":"ContainerStarted","Data":"44ebf73454d30e4300f0b72f13802be0fd92e77637080f42e6b7100ef01a6c0e"}
Sep 09 12:20:30.700478 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:30.700441    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:31.926413 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:31.926375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:32.002419 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:32.002387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:32.101077 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:32.101037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:32.339342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:32.339314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-x5zkm" event={"ID":"e1241c5f-6380-4ffe-aabd-2e50f0bbbee3","Type":"ContainerStarted","Data":"e560169296ecceb531a5e4a0e06ef48b65b18d15b50bb933a120644a39d5e5b2"}
Sep 09 12:20:32.356566 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:32.356482    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-x5zkm" podStartSLOduration=2.662566833 podStartE2EDuration="5.356462807s" totalImagesPullingTime="2.693895974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:32.355961342 +0000 UTC m=+48.680181876" watchObservedRunningTime="2026-09-09 12:20:32.356462807 +0000 UTC m=+48.680683339"
Sep 09 12:20:32.962607 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:32.962572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:32.980916 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:32.980885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:32.981083 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:32.980941    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:32.981083 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:32.981062    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:32.981189 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:32.981130    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls podName:d4f4314f-3c74-415b-8223-05252a5b04db nodeName:}" failed. No retries permitted until 2026-09-09 12:20:40.981111071 +0000 UTC m=+57.305331594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftk7f" (UID: "d4f4314f-3c74-415b-8223-05252a5b04db") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:20:32.981515 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:32.981496    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4f4314f-3c74-415b-8223-05252a5b04db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:33.065480 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:33.065448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:33.161539 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:33.161501    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:33.387818 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:33.387724    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:33.488584 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:33.488552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:33.630966 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:33.630929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:33.991339 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:33.991306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:33.991809 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:33.991415    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:20:33.991809 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:20:33.991477    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls podName:0e58e54d-f883-4fbb-8ddf-b8f65dc39874 nodeName:}" failed. No retries permitted until 2026-09-09 12:20:49.991460281 +0000 UTC m=+66.315680811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls") pod "dns-default-9vslz" (UID: "0e58e54d-f883-4fbb-8ddf-b8f65dc39874") : secret "dns-default-metrics-tls" not found
Sep 09 12:20:34.111353 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:34.111315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:34.922318 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:34.922282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:35.311808 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:35.311775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:35.506955 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:35.506922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:35.711544 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:35.711517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:35.911935 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:35.911895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:36.518525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:36.518496    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:36.690044 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:36.690013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:37.105899 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:37.105866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:37.306104 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:37.306062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:37.506366 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:37.506325    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:37.563674 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:37.563622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:37.705066 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:37.705031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:37.915299 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:37.915211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:38.151528 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:38.151494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:38.252848 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:38.252816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:38.921302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:38.921268    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:39.023365 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:39.023329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:39.122629 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:39.122588    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:39.512868 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:39.512836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:39.706348 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:39.706306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:40.135918 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:40.135882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:41.014304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.014269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:41.041295 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.041261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:41.043502 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.043475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4f4314f-3c74-415b-8223-05252a5b04db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftk7f\" (UID: \"d4f4314f-3c74-415b-8223-05252a5b04db\") " pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:41.102408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.102379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ftk7f"
Sep 09 12:20:41.314063 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.313994    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:41.368979 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.368951    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ftk7f"]
Sep 09 12:20:41.371957 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:41.371932    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f4314f_3c74_415b_8223_05252a5b04db.slice/crio-02677fb3d7a5c17346d126201099a07a9dfec9b7ade319e550129109255f2c1f WatchSource:0}: Error finding container 02677fb3d7a5c17346d126201099a07a9dfec9b7ade319e550129109255f2c1f: Status 404 returned error can't find the container with id 02677fb3d7a5c17346d126201099a07a9dfec9b7ade319e550129109255f2c1f
Sep 09 12:20:41.396165 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.396136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:41.505425 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.505395    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:41.715869 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.715836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:41.906341 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:41.906308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:42.363570 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:42.363541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftk7f" event={"ID":"d4f4314f-3c74-415b-8223-05252a5b04db","Type":"ContainerStarted","Data":"720d6881fcd824162d23579f0871484b23173635e5abf8497a646d06ce2d887d"}
Sep 09 12:20:42.363944 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:42.363594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftk7f" event={"ID":"d4f4314f-3c74-415b-8223-05252a5b04db","Type":"ContainerStarted","Data":"02677fb3d7a5c17346d126201099a07a9dfec9b7ade319e550129109255f2c1f"}
Sep 09 12:20:42.711740 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:42.711700    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:43.112849 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:43.112815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:43.311303 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:43.311269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:43.367903 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:43.367835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftk7f" event={"ID":"d4f4314f-3c74-415b-8223-05252a5b04db","Type":"ContainerStarted","Data":"8dd381a820a88529241f49c011b0367fb0fbbeb6f8daf7ea8528d91d314f4c46"}
Sep 09 12:20:43.531100 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:43.531036    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:44.282962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:44.282936    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tngrb"
Sep 09 12:20:44.373869 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:44.373835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftk7f" event={"ID":"d4f4314f-3c74-415b-8223-05252a5b04db","Type":"ContainerStarted","Data":"6256eaa4e8febdd635620c5ccfd608fbfb80ddab34b321835ce276db360be809"}
Sep 09 12:20:44.392224 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:44.392167    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ftk7f" podStartSLOduration=16.749137136999998 podStartE2EDuration="19.392149714s" totalImagesPullingTime="2.643012577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:20:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:44.39122906 +0000 UTC m=+60.715449595" watchObservedRunningTime="2026-09-09 12:20:44.392149714 +0000 UTC m=+60.716370244"
Sep 09 12:20:49.801854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.801811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:49.804037 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.804018    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:20:49.814864 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.814842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/47208fbf-65e4-4cfa-b2bd-7559528bdab4-original-pull-secret\") pod \"global-pull-secret-syncer-d9ktm\" (UID: \"47208fbf-65e4-4cfa-b2bd-7559528bdab4\") " pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:49.903155 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.903119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:49.903289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.903179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:49.905974 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.905949    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:20:49.906402 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.906386    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:20:49.915621 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.915595    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:20:49.916042 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.916026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6-metrics-certs\") pod \"network-metrics-daemon-mkb58\" (UID: \"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6\") " pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:49.926326 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.926302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjm2v\" (UniqueName: \"kubernetes.io/projected/2a234ff4-faec-4a2e-8be1-a9952531718c-kube-api-access-mjm2v\") pod \"network-check-target-wfgw7\" (UID: \"2a234ff4-faec-4a2e-8be1-a9952531718c\") " pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:49.944900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.944883    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lblnd\""
Sep 09 12:20:49.950347 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.950335    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d9ktm"
Sep 09 12:20:49.953468 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.953453    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:49.958261 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.958247    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wjmxh\""
Sep 09 12:20:49.967130 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:49.967114    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mkb58"
Sep 09 12:20:50.004139 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.004104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:50.006263 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.006240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e58e54d-f883-4fbb-8ddf-b8f65dc39874-metrics-tls\") pod \"dns-default-9vslz\" (UID: \"0e58e54d-f883-4fbb-8ddf-b8f65dc39874\") " pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:50.216004 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.215973    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hsrxf\""
Sep 09 12:20:50.224699 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.224672    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:50.233065 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.233043    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d9ktm"]
Sep 09 12:20:50.236189 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:50.236164    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47208fbf_65e4_4cfa_b2bd_7559528bdab4.slice/crio-fcb703eaae518b306990bd5dd28ace867b0bac0112ee71504236790dea6c2692 WatchSource:0}: Error finding container fcb703eaae518b306990bd5dd28ace867b0bac0112ee71504236790dea6c2692: Status 404 returned error can't find the container with id fcb703eaae518b306990bd5dd28ace867b0bac0112ee71504236790dea6c2692
Sep 09 12:20:50.243729 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.243702    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wfgw7"]
Sep 09 12:20:50.247369 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:50.247346    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a234ff4_faec_4a2e_8be1_a9952531718c.slice/crio-d8cc08aa60ef5b5d30ad74bce69b45fa01444d9439fdbe572d9ae5b5cfd1685c WatchSource:0}: Error finding container d8cc08aa60ef5b5d30ad74bce69b45fa01444d9439fdbe572d9ae5b5cfd1685c: Status 404 returned error can't find the container with id d8cc08aa60ef5b5d30ad74bce69b45fa01444d9439fdbe572d9ae5b5cfd1685c
Sep 09 12:20:50.260205 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.260181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mkb58"]
Sep 09 12:20:50.264669 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:50.264621    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2c35fc_1725_46dc_88b4_0d45ae3fd6a6.slice/crio-0afd472033778f325b8606def8a4ee6e653a468c84fcaf70212e1cf411110e76 WatchSource:0}: Error finding container 0afd472033778f325b8606def8a4ee6e653a468c84fcaf70212e1cf411110e76: Status 404 returned error can't find the container with id 0afd472033778f325b8606def8a4ee6e653a468c84fcaf70212e1cf411110e76
Sep 09 12:20:50.391742 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.391668    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkb58" event={"ID":"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6","Type":"ContainerStarted","Data":"0afd472033778f325b8606def8a4ee6e653a468c84fcaf70212e1cf411110e76"}
Sep 09 12:20:50.392812 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.392789    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d9ktm" event={"ID":"47208fbf-65e4-4cfa-b2bd-7559528bdab4","Type":"ContainerStarted","Data":"fcb703eaae518b306990bd5dd28ace867b0bac0112ee71504236790dea6c2692"}
Sep 09 12:20:50.393809 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.393780    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wfgw7" event={"ID":"2a234ff4-faec-4a2e-8be1-a9952531718c","Type":"ContainerStarted","Data":"d8cc08aa60ef5b5d30ad74bce69b45fa01444d9439fdbe572d9ae5b5cfd1685c"}
Sep 09 12:20:50.515101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:50.515073    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9vslz"]
Sep 09 12:20:50.518249 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:50.518227    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e58e54d_f883_4fbb_8ddf_b8f65dc39874.slice/crio-45348fa95d91dfe68a8014cd898e8386c98363128b55cd69c50d5606d5fcf80d WatchSource:0}: Error finding container 45348fa95d91dfe68a8014cd898e8386c98363128b55cd69c50d5606d5fcf80d: Status 404 returned error can't find the container with id 45348fa95d91dfe68a8014cd898e8386c98363128b55cd69c50d5606d5fcf80d
Sep 09 12:20:51.095894 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.095857    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"]
Sep 09 12:20:51.097678 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.097651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"
Sep 09 12:20:51.100168 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.100141    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 12:20:51.101082 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.101059    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-nqxxc\""
Sep 09 12:20:51.102332 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.102306    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 09 12:20:51.102612 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.102592    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 12:20:51.104896 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.104872    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 12:20:51.106433 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.105967    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"]
Sep 09 12:20:51.108225 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.108155    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.110879 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.110862    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 09 12:20:51.112273 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.112252    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 09 12:20:51.112509 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.112494    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 09 12:20:51.112764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.112746    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 09 12:20:51.114182 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.114162    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"]
Sep 09 12:20:51.131262 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.131238    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"]
Sep 09 12:20:51.204530 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.204493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:51.211186 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.211145    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx7r2\" (UniqueName: \"kubernetes.io/projected/17695a0b-f6d2-4c33-9840-5aa1454cb0d0-kube-api-access-cx7r2\") pod \"managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr\" (UID: \"17695a0b-f6d2-4c33-9840-5aa1454cb0d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"
Sep 09 12:20:51.211332 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.211203    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.211332 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.211232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17695a0b-f6d2-4c33-9840-5aa1454cb0d0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr\" (UID: \"17695a0b-f6d2-4c33-9840-5aa1454cb0d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"
Sep 09 12:20:51.211332 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.211256    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-ca\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.211332 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.211282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.211549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.211396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18b6749-99d1-400d-934d-7513c2b69899-additional-service-ca\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.211549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.211451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd497\" (UniqueName: \"kubernetes.io/projected/a18b6749-99d1-400d-934d-7513c2b69899-kube-api-access-qd497\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.211549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.211479    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-hub\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.292701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.292656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:51.312664 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.312617    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx7r2\" (UniqueName: \"kubernetes.io/projected/17695a0b-f6d2-4c33-9840-5aa1454cb0d0-kube-api-access-cx7r2\") pod \"managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr\" (UID: \"17695a0b-f6d2-4c33-9840-5aa1454cb0d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"
Sep 09 12:20:51.312838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.312687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.312838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.312711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17695a0b-f6d2-4c33-9840-5aa1454cb0d0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr\" (UID: \"17695a0b-f6d2-4c33-9840-5aa1454cb0d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"
Sep 09 12:20:51.312838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.312732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-ca\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.312838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.312747    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.312838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.312777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18b6749-99d1-400d-934d-7513c2b69899-additional-service-ca\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.312838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.312800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd497\" (UniqueName: \"kubernetes.io/projected/a18b6749-99d1-400d-934d-7513c2b69899-kube-api-access-qd497\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.312838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.312816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-hub\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.314027 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.314000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a18b6749-99d1-400d-934d-7513c2b69899-additional-service-ca\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.316091 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.316064    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-hub\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.316759 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.316716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-ca\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.317336 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.317077    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17695a0b-f6d2-4c33-9840-5aa1454cb0d0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr\" (UID: \"17695a0b-f6d2-4c33-9840-5aa1454cb0d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"
Sep 09 12:20:51.317416 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.317340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.318581 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.318560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a18b6749-99d1-400d-934d-7513c2b69899-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.322430 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.322406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx7r2\" (UniqueName: \"kubernetes.io/projected/17695a0b-f6d2-4c33-9840-5aa1454cb0d0-kube-api-access-cx7r2\") pod \"managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr\" (UID: \"17695a0b-f6d2-4c33-9840-5aa1454cb0d0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"
Sep 09 12:20:51.322967 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.322921    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd497\" (UniqueName: \"kubernetes.io/projected/a18b6749-99d1-400d-934d-7513c2b69899-kube-api-access-qd497\") pod \"cluster-proxy-proxy-agent-595bccdfc8-zvxtd\" (UID: \"a18b6749-99d1-400d-934d-7513c2b69899\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.391696 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.391603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:51.399401 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.399363    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9vslz" event={"ID":"0e58e54d-f883-4fbb-8ddf-b8f65dc39874","Type":"ContainerStarted","Data":"45348fa95d91dfe68a8014cd898e8386c98363128b55cd69c50d5606d5fcf80d"}
Sep 09 12:20:51.419841 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.419813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"
Sep 09 12:20:51.428859 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.428824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"
Sep 09 12:20:51.491568 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.491504    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:51.703479 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.703286    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr"]
Sep 09 12:20:51.706405 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:51.706375    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17695a0b_f6d2_4c33_9840_5aa1454cb0d0.slice/crio-35d6af18a6f52b4e9cdfefc38b0850a04a7529fdc2631568f874f80c9784b04a WatchSource:0}: Error finding container 35d6af18a6f52b4e9cdfefc38b0850a04a7529fdc2631568f874f80c9784b04a: Status 404 returned error can't find the container with id 35d6af18a6f52b4e9cdfefc38b0850a04a7529fdc2631568f874f80c9784b04a
Sep 09 12:20:51.713064 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:51.713005    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd"]
Sep 09 12:20:51.717105 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:20:51.717076    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda18b6749_99d1_400d_934d_7513c2b69899.slice/crio-00eb1d8030e697b05470fb0c66c78ce6ede2e2b2b413a0a93d60c327cd7e03e6 WatchSource:0}: Error finding container 00eb1d8030e697b05470fb0c66c78ce6ede2e2b2b413a0a93d60c327cd7e03e6: Status 404 returned error can't find the container with id 00eb1d8030e697b05470fb0c66c78ce6ede2e2b2b413a0a93d60c327cd7e03e6
Sep 09 12:20:52.405578 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:52.405538    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr" event={"ID":"17695a0b-f6d2-4c33-9840-5aa1454cb0d0","Type":"ContainerStarted","Data":"35d6af18a6f52b4e9cdfefc38b0850a04a7529fdc2631568f874f80c9784b04a"}
Sep 09 12:20:52.407916 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:52.407883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd" event={"ID":"a18b6749-99d1-400d-934d-7513c2b69899","Type":"ContainerStarted","Data":"00eb1d8030e697b05470fb0c66c78ce6ede2e2b2b413a0a93d60c327cd7e03e6"}
Sep 09 12:20:52.618004 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:52.617931    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:52.720522 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:52.720484    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:52.808044 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:52.808009    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:52.874630 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:52.874593    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:53.475036 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:53.475001    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:55.860243 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:55.860167    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:55.941044 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:55.941009    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:56.013983 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:56.013938    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:58.051238 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.051206    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:58.431998 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.431957    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd" event={"ID":"a18b6749-99d1-400d-934d-7513c2b69899","Type":"ContainerStarted","Data":"abbe9168293b415daeb9447d01af55b0b285c1308ca01addaf606c1dcc174b2b"}
Sep 09 12:20:58.432145 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.432009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd" event={"ID":"a18b6749-99d1-400d-934d-7513c2b69899","Type":"ContainerStarted","Data":"de6bbcb86a0b83b424ad35c4c0ec88535bba8cd2a2fecff22ee6ace6bb8d7fdc"}
Sep 09 12:20:58.432145 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.432024    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd" event={"ID":"a18b6749-99d1-400d-934d-7513c2b69899","Type":"ContainerStarted","Data":"377ea8e0847db7453cc4d75b26e1528567eb70d05a013d6ed658a2ac63ff650d"}
Sep 09 12:20:58.433540 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.433510    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wfgw7" event={"ID":"2a234ff4-faec-4a2e-8be1-a9952531718c","Type":"ContainerStarted","Data":"f01a86538115eae99b04e50fe9dbef6d0899c8a2c66d794216e155a7718bda1f"}
Sep 09 12:20:58.433698 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.433589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:20:58.435244 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.435213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9vslz" event={"ID":"0e58e54d-f883-4fbb-8ddf-b8f65dc39874","Type":"ContainerStarted","Data":"4cf2bc63c22d14cec2224671014001b2408815e0f702d6ab750650a953139a0e"}
Sep 09 12:20:58.435347 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.435266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9vslz"
Sep 09 12:20:58.435347 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.435292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9vslz" event={"ID":"0e58e54d-f883-4fbb-8ddf-b8f65dc39874","Type":"ContainerStarted","Data":"b0d9319052543ce046d30244aa1edf389ed2cce3fac9ec290b49876d06c0de02"}
Sep 09 12:20:58.436756 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.436737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr" event={"ID":"17695a0b-f6d2-4c33-9840-5aa1454cb0d0","Type":"ContainerStarted","Data":"502963d796af1da00ffc7a12a2d7870f4f38a863186ada8d0b245f403576e69e"}
Sep 09 12:20:58.438468 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.438448    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkb58" event={"ID":"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6","Type":"ContainerStarted","Data":"41da5721818a39a70f44ddaf93c48eaff09df7c848608ba7e975f59d0c35af65"}
Sep 09 12:20:58.438578 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.438501    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mkb58" event={"ID":"9d2c35fc-1725-46dc-88b4-0d45ae3fd6a6","Type":"ContainerStarted","Data":"6cc7893dff3864460b180a6ab81ff7a10e021951d4f7b24a73e054e7beef1ef1"}
Sep 09 12:20:58.439836 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.439816    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d9ktm" event={"ID":"47208fbf-65e4-4cfa-b2bd-7559528bdab4","Type":"ContainerStarted","Data":"1817e31a5a638c69ad1bd989b8c05137aafb834e0f3ff2b679c139ebc6aca58f"}
Sep 09 12:20:58.450476 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.450435    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-595bccdfc8-zvxtd" podStartSLOduration=1.091059843 podStartE2EDuration="7.450423231s" totalImagesPullingTime="6.359363388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:58.449074993 +0000 UTC m=+74.773295525" watchObservedRunningTime="2026-09-09 12:20:58.450423231 +0000 UTC m=+74.774643793"
Sep 09 12:20:58.464007 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.463967    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wfgw7" podStartSLOduration=66.686793694 podStartE2EDuration="1m14.463953502s" totalImagesPullingTime="7.777159808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:58.463334911 +0000 UTC m=+74.787555443" watchObservedRunningTime="2026-09-09 12:20:58.463953502 +0000 UTC m=+74.788174034"
Sep 09 12:20:58.478507 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.478460    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7dddd6db4b-gfdzr" podStartSLOduration=1.117167786 podStartE2EDuration="7.478446669s" totalImagesPullingTime="6.361278883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:58.478130366 +0000 UTC m=+74.802350890" watchObservedRunningTime="2026-09-09 12:20:58.478446669 +0000 UTC m=+74.802667199"
Sep 09 12:20:58.491402 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.491365    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d9ktm" podStartSLOduration=66.669990705 podStartE2EDuration="1m14.491353574s" totalImagesPullingTime="7.821362869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:58.491077032 +0000 UTC m=+74.815297564" watchObservedRunningTime="2026-09-09 12:20:58.491353574 +0000 UTC m=+74.815574156"
Sep 09 12:20:58.505989 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.505948    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mkb58" podStartSLOduration=66.74512411 podStartE2EDuration="1m14.505936953s" totalImagesPullingTime="7.760812843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:58.505718046 +0000 UTC m=+74.829938569" watchObservedRunningTime="2026-09-09 12:20:58.505936953 +0000 UTC m=+74.830157483"
Sep 09 12:20:58.528392 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.528357    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9vslz" podStartSLOduration=34.093788052 podStartE2EDuration="40.528346285s" totalImagesPullingTime="6.434558233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:20:58.527898493 +0000 UTC m=+74.852119025" watchObservedRunningTime="2026-09-09 12:20:58.528346285 +0000 UTC m=+74.852566853"
Sep 09 12:20:58.777662 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.777599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:58.877792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.877755    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:58.980918 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:58.980878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.079271 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:59.079191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.158758 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:59.158727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.275855 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:59.275820    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.459151 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:59.459118    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.650172 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:59.650133    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:20:59.713788 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:20:59.713707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:00.058211 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:00.058135    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:00.117134 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:00.117088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:00.251217 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:00.251171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:00.866830 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:00.866799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:00.974499 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:00.974463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:01.068237 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.068205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:01.221735 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.221699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:01.292361 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.291052    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vvh4h"]
Sep 09 12:21:01.299315 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.299287    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.301403 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.301373    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 12:21:01.301403 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.301394    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 12:21:01.301725 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.301706    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 12:21:01.302091 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.302077    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 12:21:01.302496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.302463    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tnpqb\""
Sep 09 12:21:01.302616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.302505    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 12:21:01.302616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.302470    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 12:21:01.308494 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.308472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:01.392881 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.392848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-wtmp\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.392881 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.392885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-sys\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.393102 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.392928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.393102 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.393001    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-textfile\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.393102 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.393061    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2pmp\" (UniqueName: \"kubernetes.io/projected/4f82eed3-ba39-4e0f-a052-b40c22497062-kube-api-access-g2pmp\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.393232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.393100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-tls\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.393232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.393136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.393232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.393162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-root\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.393369 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.393264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f82eed3-ba39-4e0f-a052-b40c22497062-metrics-client-ca\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.493701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.493701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-root\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.493897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493706    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f82eed3-ba39-4e0f-a052-b40c22497062-metrics-client-ca\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.493897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-wtmp\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.493897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493766    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-root\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.493897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-wtmp\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.493897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-sys\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.494126 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f82eed3-ba39-4e0f-a052-b40c22497062-sys\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.494126 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.493997    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.494126 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.494051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-textfile\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.494473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.494092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2pmp\" (UniqueName: \"kubernetes.io/projected/4f82eed3-ba39-4e0f-a052-b40c22497062-kube-api-access-g2pmp\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.494473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.494239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-tls\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.494473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.494297    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-accelerators-collector-config\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.494473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.494309    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f82eed3-ba39-4e0f-a052-b40c22497062-metrics-client-ca\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.494473 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:21:01.494354    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 12:21:01.494473 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:21:01.494412    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-tls podName:4f82eed3-ba39-4e0f-a052-b40c22497062 nodeName:}" failed. No retries permitted until 2026-09-09 12:21:01.99439254 +0000 UTC m=+78.318613053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-tls") pod "node-exporter-vvh4h" (UID: "4f82eed3-ba39-4e0f-a052-b40c22497062") : secret "node-exporter-tls" not found
Sep 09 12:21:01.494473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.494414    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-textfile\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.496524 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.496493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.502242 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.502209    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2pmp\" (UniqueName: \"kubernetes.io/projected/4f82eed3-ba39-4e0f-a052-b40c22497062-kube-api-access-g2pmp\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:01.999240 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:01.999204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-tls\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:02.001490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.001467    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4f82eed3-ba39-4e0f-a052-b40c22497062-node-exporter-tls\") pod \"node-exporter-vvh4h\" (UID: \"4f82eed3-ba39-4e0f-a052-b40c22497062\") " pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:02.044734 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.044701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:02.144950 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.144901    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:02.210560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.210530    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vvh4h"
Sep 09 12:21:02.224159 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:21:02.224125    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f82eed3_ba39_4e0f_a052_b40c22497062.slice/crio-69b48800545ba95beda74579a677191fa15bbe25d2cc0ddaf14ee98da6c6598d WatchSource:0}: Error finding container 69b48800545ba95beda74579a677191fa15bbe25d2cc0ddaf14ee98da6c6598d: Status 404 returned error can't find the container with id 69b48800545ba95beda74579a677191fa15bbe25d2cc0ddaf14ee98da6c6598d
Sep 09 12:21:02.246035 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.245993    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:02.335177 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.335097    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:02.446525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.446488    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:02.457122 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.457091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvh4h" event={"ID":"4f82eed3-ba39-4e0f-a052-b40c22497062","Type":"ContainerStarted","Data":"69b48800545ba95beda74579a677191fa15bbe25d2cc0ddaf14ee98da6c6598d"}
Sep 09 12:21:02.559105 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:02.559071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:03.462206 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:03.462170    2579 generic.go:356] "Generic (PLEG): container finished" podID="4f82eed3-ba39-4e0f-a052-b40c22497062" containerID="92c393b2e6af7ad62e28ca220284d4c6cf51eb769aa6006f7615f53d16e3ceae" exitCode=0
Sep 09 12:21:03.462549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:03.462253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvh4h" event={"ID":"4f82eed3-ba39-4e0f-a052-b40c22497062","Type":"ContainerDied","Data":"92c393b2e6af7ad62e28ca220284d4c6cf51eb769aa6006f7615f53d16e3ceae"}
Sep 09 12:21:04.194309 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:04.194278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:04.298048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:04.297994    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:04.394127 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:04.394087    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:04.458239 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:04.458193    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:04.467685 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:04.467653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvh4h" event={"ID":"4f82eed3-ba39-4e0f-a052-b40c22497062","Type":"ContainerStarted","Data":"66a3cdaa5e16eb66bfaa96c3a4785e92c0be042e2e1c7b1b2be262b7d37dbd5e"}
Sep 09 12:21:04.468102 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:04.467705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vvh4h" event={"ID":"4f82eed3-ba39-4e0f-a052-b40c22497062","Type":"ContainerStarted","Data":"1a7e872b8b1b0eced86589b7de0fac67a2708d9aad9367845a0ebace4e784caf"}
Sep 09 12:21:06.010918 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:06.010875    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:06.099874 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:06.099834    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:06.436802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:06.436608    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:07.536125 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:07.536085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:08.445702 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:08.445672    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9vslz"
Sep 09 12:21:08.471455 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:08.471393    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vvh4h" podStartSLOduration=6.353135415 podStartE2EDuration="7.471373808s" totalImagesPullingTime="1.057360492s" totalInitContainerRuntime="60.877901ms" isStatefulPod=true podCreationTimestamp="2026-09-09 12:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:21:04.488925873 +0000 UTC m=+80.813146405" watchObservedRunningTime="2026-09-09 12:21:08.471373808 +0000 UTC m=+84.795594339"
Sep 09 12:21:09.828409 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:09.828375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:09.889827 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:09.889791    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:12.253579 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:12.253537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:12.355072 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:12.355018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:14.253252 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:14.253211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:14.323853 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:14.323823    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:14.733027 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:14.732990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:15.588897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:15.588857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:15.660286 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:15.660254    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:18.217911 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:18.217882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:18.302234 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:18.302201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:18.381126 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:18.381094    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:21.270411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:21.270375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:21.373059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:21.373024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:23.489508 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:23.489470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:23.562952 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:23.562922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:24.665365 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:24.665327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:24.766334 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:24.766296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:28.053560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:28.053528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:28.112253 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:28.112212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:28.491814 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:28.491769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:28.556750 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:28.556721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:29.365118 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:29.365088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:29.445571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:29.445545    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wfgw7"
Sep 09 12:21:30.197561 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:30.197516    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:40.677707 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:40.677671    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j46h4_949d2d6e-2a7a-4d2b-9d52-9303f3540d23/serve-healthcheck-canary/0.log"
Sep 09 12:21:43.061457 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:43.061422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:43.161691 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:43.161626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:50.173404 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:50.173363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:50.274471 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:50.274436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:21:50.336140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:21:50.336102    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:20.490766 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:22:20.490721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:20.714794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:22:20.714755    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:30.086069 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:22:30.086036    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:40.816915 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:22:40.816820    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:40.918332 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:22:40.918290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:22:44.956355 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:22:44.956323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:10.740166 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:10.740105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:13.831866 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:13.831824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:35.740272 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:35.740235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:35.840180 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:35.840144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:47.723920 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:47.723884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:47.825848 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:47.825810    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:58.052647 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:58.052605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:58.153548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:58.153511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:59.073363 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:59.073314    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:59.187721 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:59.187684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:59.252981 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:59.252941    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:23:59.996371 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:23:59.996338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:01.862563 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:24:01.862526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:10.091919 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:24:10.091831    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:10.156052 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:24:10.156015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:10.212923 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:24:10.212883    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:44.008315 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:24:44.008277    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 12:24:45.019082 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:24:45.019049    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:46.385165 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:24:46.385128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:24:51.812264 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:24:51.812225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:06.477763 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:06.477721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:07.857898 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:07.857866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:10.087511 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:10.087473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:10.704775 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:10.704737    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:11.556549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:11.556514    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:11.660894 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:11.660859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:11.963728 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:11.963694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:12.066145 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.066106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:12.528038 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.527998    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"]
Sep 09 12:25:12.530993 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.530976    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.533223 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.533198    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:25:12.533909 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.533893    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sghgc\""
Sep 09 12:25:12.533997 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.533917    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:25:12.540108 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.540087    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"]
Sep 09 12:25:12.586757 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.586725    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.587156 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.586763    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z45wn\" (UniqueName: \"kubernetes.io/projected/9d4c5be1-4a49-4b33-aabf-fc67eb571081-kube-api-access-z45wn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.587156 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.586822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.687705 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.687670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.687705 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.687710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z45wn\" (UniqueName: \"kubernetes.io/projected/9d4c5be1-4a49-4b33-aabf-fc67eb571081-kube-api-access-z45wn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.687943 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.687739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.688139 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.688122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.688203 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.688122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.705518 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.705492    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z45wn\" (UniqueName: \"kubernetes.io/projected/9d4c5be1-4a49-4b33-aabf-fc67eb571081-kube-api-access-z45wn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:12.840174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:12.840083    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:13.109701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:13.109673    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"]
Sep 09 12:25:13.112177 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:25:13.112151    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4c5be1_4a49_4b33_aabf_fc67eb571081.slice/crio-b3227b1004767e14ce83d40d351d1cf2a0e39a20cf6b9c29e979dd0032fc15b9 WatchSource:0}: Error finding container b3227b1004767e14ce83d40d351d1cf2a0e39a20cf6b9c29e979dd0032fc15b9: Status 404 returned error can't find the container with id b3227b1004767e14ce83d40d351d1cf2a0e39a20cf6b9c29e979dd0032fc15b9
Sep 09 12:25:13.114084 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:13.114067    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:25:13.272864 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:13.272832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg" event={"ID":"9d4c5be1-4a49-4b33-aabf-fc67eb571081","Type":"ContainerStarted","Data":"b3227b1004767e14ce83d40d351d1cf2a0e39a20cf6b9c29e979dd0032fc15b9"}
Sep 09 12:25:14.380984 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:14.380948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:14.594911 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:14.594846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:16.873933 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:16.873891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:16.974768 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:16.974723    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:19.299013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:19.298979    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerID="d2f0d14dd321d76b90685cf67b874cd3630812db9f4f870fca2e2dd93d5aebde" exitCode=0
Sep 09 12:25:19.299386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:19.299061    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg" event={"ID":"9d4c5be1-4a49-4b33-aabf-fc67eb571081","Type":"ContainerDied","Data":"d2f0d14dd321d76b90685cf67b874cd3630812db9f4f870fca2e2dd93d5aebde"}
Sep 09 12:25:20.013960 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:20.013917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:20.074770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:20.074720    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:21.308714 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:21.308674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg" event={"ID":"9d4c5be1-4a49-4b33-aabf-fc67eb571081","Type":"ContainerStarted","Data":"1d802682cfce2fea8ddd27b77523f3294c439e48d3d91ddd0b4e55ef6ea9d22c"}
Sep 09 12:25:22.016745 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:22.016707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:22.313764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:22.313734    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerID="1d802682cfce2fea8ddd27b77523f3294c439e48d3d91ddd0b4e55ef6ea9d22c" exitCode=0
Sep 09 12:25:22.314068 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:22.313792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg" event={"ID":"9d4c5be1-4a49-4b33-aabf-fc67eb571081","Type":"ContainerDied","Data":"1d802682cfce2fea8ddd27b77523f3294c439e48d3d91ddd0b4e55ef6ea9d22c"}
Sep 09 12:25:26.214578 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:26.214535    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:26.303418 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:26.303379    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:30.343721 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:30.343683    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerID="6f3d68d8c579a6885bd383ff70c21222630849001b35f6fa4e58a0d8de906bd1" exitCode=0
Sep 09 12:25:30.344162 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:30.343776    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg" event={"ID":"9d4c5be1-4a49-4b33-aabf-fc67eb571081","Type":"ContainerDied","Data":"6f3d68d8c579a6885bd383ff70c21222630849001b35f6fa4e58a0d8de906bd1"}
Sep 09 12:25:31.469058 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.469036    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:31.531018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.530990    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z45wn\" (UniqueName: \"kubernetes.io/projected/9d4c5be1-4a49-4b33-aabf-fc67eb571081-kube-api-access-z45wn\") pod \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") "
Sep 09 12:25:31.531173 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.531030    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-bundle\") pod \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") "
Sep 09 12:25:31.531173 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.531052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-util\") pod \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\" (UID: \"9d4c5be1-4a49-4b33-aabf-fc67eb571081\") "
Sep 09 12:25:31.531448 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.531425    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-bundle" (OuterVolumeSpecName: "bundle") pod "9d4c5be1-4a49-4b33-aabf-fc67eb571081" (UID: "9d4c5be1-4a49-4b33-aabf-fc67eb571081"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:31.533206 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.533178    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d4c5be1-4a49-4b33-aabf-fc67eb571081-kube-api-access-z45wn" (OuterVolumeSpecName: "kube-api-access-z45wn") pod "9d4c5be1-4a49-4b33-aabf-fc67eb571081" (UID: "9d4c5be1-4a49-4b33-aabf-fc67eb571081"). InnerVolumeSpecName "kube-api-access-z45wn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:25:31.535596 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.535571    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-util" (OuterVolumeSpecName: "util") pod "9d4c5be1-4a49-4b33-aabf-fc67eb571081" (UID: "9d4c5be1-4a49-4b33-aabf-fc67eb571081"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:25:31.631728 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.631621    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-util\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:25:31.631728 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.631678    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z45wn\" (UniqueName: \"kubernetes.io/projected/9d4c5be1-4a49-4b33-aabf-fc67eb571081-kube-api-access-z45wn\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:25:31.631728 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:31.631689    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d4c5be1-4a49-4b33-aabf-fc67eb571081-bundle\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:25:32.351978 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:32.351943    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg" event={"ID":"9d4c5be1-4a49-4b33-aabf-fc67eb571081","Type":"ContainerDied","Data":"b3227b1004767e14ce83d40d351d1cf2a0e39a20cf6b9c29e979dd0032fc15b9"}
Sep 09 12:25:32.351978 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:32.351971    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8v2wg"
Sep 09 12:25:32.351978 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:32.351985    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3227b1004767e14ce83d40d351d1cf2a0e39a20cf6b9c29e979dd0032fc15b9"
Sep 09 12:25:34.600125 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:34.600085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:34.699837 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:34.699784    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:36.587798 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.587713    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"]
Sep 09 12:25:36.588197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.587962    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerName="util"
Sep 09 12:25:36.588197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.587973    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerName="util"
Sep 09 12:25:36.588197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.587980    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerName="extract"
Sep 09 12:25:36.588197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.587986    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerName="extract"
Sep 09 12:25:36.588197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.587996    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerName="pull"
Sep 09 12:25:36.588197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.588002    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerName="pull"
Sep 09 12:25:36.588197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.588049    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d4c5be1-4a49-4b33-aabf-fc67eb571081" containerName="extract"
Sep 09 12:25:36.590727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.590709    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"
Sep 09 12:25:36.593398 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.593376    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gjplj\""
Sep 09 12:25:36.594247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.594223    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:25:36.594323 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.594237    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 12:25:36.599130 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.599102    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"]
Sep 09 12:25:36.669286 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.669246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j75qz\" (UniqueName: \"kubernetes.io/projected/8c915815-3a94-4eb3-8446-72a02f8d6015-kube-api-access-j75qz\") pod \"openshift-lws-operator-fd8ccff4c-fxvzd\" (UID: \"8c915815-3a94-4eb3-8446-72a02f8d6015\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"
Sep 09 12:25:36.669461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.669320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c915815-3a94-4eb3-8446-72a02f8d6015-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fxvzd\" (UID: \"8c915815-3a94-4eb3-8446-72a02f8d6015\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"
Sep 09 12:25:36.770129 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.770093    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c915815-3a94-4eb3-8446-72a02f8d6015-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fxvzd\" (UID: \"8c915815-3a94-4eb3-8446-72a02f8d6015\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"
Sep 09 12:25:36.770273 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.770146    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j75qz\" (UniqueName: \"kubernetes.io/projected/8c915815-3a94-4eb3-8446-72a02f8d6015-kube-api-access-j75qz\") pod \"openshift-lws-operator-fd8ccff4c-fxvzd\" (UID: \"8c915815-3a94-4eb3-8446-72a02f8d6015\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"
Sep 09 12:25:36.770468 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.770448    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c915815-3a94-4eb3-8446-72a02f8d6015-tmp\") pod \"openshift-lws-operator-fd8ccff4c-fxvzd\" (UID: \"8c915815-3a94-4eb3-8446-72a02f8d6015\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"
Sep 09 12:25:36.778101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.778070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j75qz\" (UniqueName: \"kubernetes.io/projected/8c915815-3a94-4eb3-8446-72a02f8d6015-kube-api-access-j75qz\") pod \"openshift-lws-operator-fd8ccff4c-fxvzd\" (UID: \"8c915815-3a94-4eb3-8446-72a02f8d6015\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"
Sep 09 12:25:36.899908 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:36.899823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"
Sep 09 12:25:37.168562 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:37.168490    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd"]
Sep 09 12:25:37.171463 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:25:37.171437    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c915815_3a94_4eb3_8446_72a02f8d6015.slice/crio-291620c1abc30fc810ab8117cb3d29ba968e7495dc745a4e189b97d02a08f3a6 WatchSource:0}: Error finding container 291620c1abc30fc810ab8117cb3d29ba968e7495dc745a4e189b97d02a08f3a6: Status 404 returned error can't find the container with id 291620c1abc30fc810ab8117cb3d29ba968e7495dc745a4e189b97d02a08f3a6
Sep 09 12:25:37.370875 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:37.370840    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd" event={"ID":"8c915815-3a94-4eb3-8446-72a02f8d6015","Type":"ContainerStarted","Data":"291620c1abc30fc810ab8117cb3d29ba968e7495dc745a4e189b97d02a08f3a6"}
Sep 09 12:25:41.387713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:41.387673    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd" event={"ID":"8c915815-3a94-4eb3-8446-72a02f8d6015","Type":"ContainerStarted","Data":"92c22ae939084fc428661eaab433f69fab453da6aeef1370a265dc3193587f19"}
Sep 09 12:25:41.405990 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:41.405932    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-fxvzd" podStartSLOduration=1.9153952730000001 podStartE2EDuration="5.405912185s" totalImagesPullingTime="3.490516912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:41.404892301 +0000 UTC m=+357.729112834" watchObservedRunningTime="2026-09-09 12:25:41.405912185 +0000 UTC m=+357.730132715"
Sep 09 12:25:43.837534 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:43.837496    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:43.941647 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:43.941593    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:48.008351 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.008304    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:48.072622 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.072590    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"]
Sep 09 12:25:48.075458 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.075428    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:25:48.075869 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.075851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.079264 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.079244    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 12:25:48.079355 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.079253    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5rjzx\""
Sep 09 12:25:48.079426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.079267    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 12:25:48.079426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.079300    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 12:25:48.084078 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.084058    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"]
Sep 09 12:25:48.149893 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.149857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a3e452e-2d73-44de-8a71-0f706c9ad678-cert\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.150065 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.149914    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kllkv\" (UniqueName: \"kubernetes.io/projected/7a3e452e-2d73-44de-8a71-0f706c9ad678-kube-api-access-kllkv\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.150065 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.149978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a3e452e-2d73-44de-8a71-0f706c9ad678-manager-config\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.150065 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.150043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3e452e-2d73-44de-8a71-0f706c9ad678-metrics-cert\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.250426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.250391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kllkv\" (UniqueName: \"kubernetes.io/projected/7a3e452e-2d73-44de-8a71-0f706c9ad678-kube-api-access-kllkv\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.250426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.250429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a3e452e-2d73-44de-8a71-0f706c9ad678-manager-config\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.250648 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.250449    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3e452e-2d73-44de-8a71-0f706c9ad678-metrics-cert\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.250648 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.250472    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a3e452e-2d73-44de-8a71-0f706c9ad678-cert\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.251151 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.251129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a3e452e-2d73-44de-8a71-0f706c9ad678-manager-config\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.252923 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.252904    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a3e452e-2d73-44de-8a71-0f706c9ad678-cert\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.253008 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.252952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3e452e-2d73-44de-8a71-0f706c9ad678-metrics-cert\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.259361 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.259306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kllkv\" (UniqueName: \"kubernetes.io/projected/7a3e452e-2d73-44de-8a71-0f706c9ad678-kube-api-access-kllkv\") pod \"lws-controller-manager-5465bf697c-g486n\" (UID: \"7a3e452e-2d73-44de-8a71-0f706c9ad678\") " pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.385392 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.385352    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:48.656354 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:48.656264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"]
Sep 09 12:25:48.659938 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:25:48.659912    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3e452e_2d73_44de_8a71_0f706c9ad678.slice/crio-133a5b2d65118b7fcec9c158ec569e714397f33cf404d682f59296ae6ac8344b WatchSource:0}: Error finding container 133a5b2d65118b7fcec9c158ec569e714397f33cf404d682f59296ae6ac8344b: Status 404 returned error can't find the container with id 133a5b2d65118b7fcec9c158ec569e714397f33cf404d682f59296ae6ac8344b
Sep 09 12:25:49.419180 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:49.419144    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n" event={"ID":"7a3e452e-2d73-44de-8a71-0f706c9ad678","Type":"ContainerStarted","Data":"133a5b2d65118b7fcec9c158ec569e714397f33cf404d682f59296ae6ac8344b"}
Sep 09 12:25:52.431987 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:52.431892    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n" event={"ID":"7a3e452e-2d73-44de-8a71-0f706c9ad678","Type":"ContainerStarted","Data":"31a11778c5ca7ce3e9e7b0c03a92014616b535a9b885694c91eae9cfcec71403"}
Sep 09 12:25:52.432379 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:52.432007    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:25:52.450150 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:25:52.450105    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n" podStartSLOduration=1.033720407 podStartE2EDuration="4.450087068s" totalImagesPullingTime="3.416366661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:25:52.449115552 +0000 UTC m=+368.773336083" watchObservedRunningTime="2026-09-09 12:25:52.450087068 +0000 UTC m=+368.774307599"
Sep 09 12:26:02.555894 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:02.555850    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:03.437925 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.437896    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5465bf697c-g486n"
Sep 09 12:26:03.595422 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.595386    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"]
Sep 09 12:26:03.599341 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.599320    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.602364 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.602344    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4l7r7\""
Sep 09 12:26:03.602490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.602367    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 12:26:03.602490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.602341    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 12:26:03.602490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.602351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 12:26:03.602490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.602478    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 12:26:03.615150 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.615126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"]
Sep 09 12:26:03.756858 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.756826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljnvh\" (UniqueName: \"kubernetes.io/projected/e589d913-a2d5-42cb-beef-deb4194efcdc-kube-api-access-ljnvh\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.757014 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.756873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e589d913-a2d5-42cb-beef-deb4194efcdc-tmp\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.757014 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.756902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e589d913-a2d5-42cb-beef-deb4194efcdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.757014 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.756975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e589d913-a2d5-42cb-beef-deb4194efcdc-manifests\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.757118 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.757017    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e589d913-a2d5-42cb-beef-deb4194efcdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.857465 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.857423    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e589d913-a2d5-42cb-beef-deb4194efcdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.857694 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.857480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljnvh\" (UniqueName: \"kubernetes.io/projected/e589d913-a2d5-42cb-beef-deb4194efcdc-kube-api-access-ljnvh\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.857694 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.857516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e589d913-a2d5-42cb-beef-deb4194efcdc-tmp\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.857694 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.857672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e589d913-a2d5-42cb-beef-deb4194efcdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.857869 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.857730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e589d913-a2d5-42cb-beef-deb4194efcdc-manifests\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.858025 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.858003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e589d913-a2d5-42cb-beef-deb4194efcdc-manifests\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.859907 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.859875    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e589d913-a2d5-42cb-beef-deb4194efcdc-tmp\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.860055 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.860040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e589d913-a2d5-42cb-beef-deb4194efcdc-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.860144 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.860128    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e589d913-a2d5-42cb-beef-deb4194efcdc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.866378 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.866351    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljnvh\" (UniqueName: \"kubernetes.io/projected/e589d913-a2d5-42cb-beef-deb4194efcdc-kube-api-access-ljnvh\") pod \"opendatahub-operator-controller-manager-6c5d49f968-vmmnf\" (UID: \"e589d913-a2d5-42cb-beef-deb4194efcdc\") " pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:03.909216 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:03.909177    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:04.129458 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:04.129372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:04.185497 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:04.185409    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"]
Sep 09 12:26:04.188319 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:26:04.188295    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode589d913_a2d5_42cb_beef_deb4194efcdc.slice/crio-5e25a502e52586899c8d15b3b2dc99f9c096aed16a57d92a4da77d6c90360a20 WatchSource:0}: Error finding container 5e25a502e52586899c8d15b3b2dc99f9c096aed16a57d92a4da77d6c90360a20: Status 404 returned error can't find the container with id 5e25a502e52586899c8d15b3b2dc99f9c096aed16a57d92a4da77d6c90360a20
Sep 09 12:26:04.279481 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:04.279445    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:04.473387 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:04.473349    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf" event={"ID":"e589d913-a2d5-42cb-beef-deb4194efcdc","Type":"ContainerStarted","Data":"5e25a502e52586899c8d15b3b2dc99f9c096aed16a57d92a4da77d6c90360a20"}
Sep 09 12:26:09.494490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:09.494452    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf" event={"ID":"e589d913-a2d5-42cb-beef-deb4194efcdc","Type":"ContainerStarted","Data":"ab3bdbfad235425a6c1f3a1ee66253e6027eb7296a7474ffef395995bc441799"}
Sep 09 12:26:10.499684 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:10.499621    2579 generic.go:356] "Generic (PLEG): container finished" podID="e589d913-a2d5-42cb-beef-deb4194efcdc" containerID="ab3bdbfad235425a6c1f3a1ee66253e6027eb7296a7474ffef395995bc441799" exitCode=0
Sep 09 12:26:10.500101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:10.499708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf" event={"ID":"e589d913-a2d5-42cb-beef-deb4194efcdc","Type":"ContainerDied","Data":"ab3bdbfad235425a6c1f3a1ee66253e6027eb7296a7474ffef395995bc441799"}
Sep 09 12:26:12.508968 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:12.508926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf" event={"ID":"e589d913-a2d5-42cb-beef-deb4194efcdc","Type":"ContainerStarted","Data":"f6410ea486d3ca4e2700bc2095308c051adf68022af2c4de886438aaa4a0ed82"}
Sep 09 12:26:12.509455 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:12.509066    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:12.530897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:12.530844    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf" podStartSLOduration=2.51464323 podStartE2EDuration="9.530829288s" totalImagesPullingTime="5.929408189s" totalInitContainerRuntime="1.086777869s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:12.528991658 +0000 UTC m=+388.853212201" watchObservedRunningTime="2026-09-09 12:26:12.530829288 +0000 UTC m=+388.855049818"
Sep 09 12:26:18.447380 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:18.447345    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:18.505313 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:18.505273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:18.590820 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:18.590782    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:18.649467 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:18.649430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:19.188445 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:19.188408    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:23.514901 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:23.514870    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c5d49f968-vmmnf"
Sep 09 12:26:25.187567 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:25.187525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:25.285520 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:25.285480    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:25.424253 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:25.424187    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:25.616523 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:25.616484    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:26.939764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:26.939724    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:26.997781 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:26.997747    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:27.815507 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:27.815470    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:29.717158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:29.717120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:29.826027 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:29.825992    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:29.990737 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:29.990627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:30.388901 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:30.388815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:31.532314 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:31.532277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:31.631787 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:31.631754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:31.959980 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:31.959953    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:32.101787 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.101759    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"]
Sep 09 12:26:32.113178 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.113150    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.115628 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.115601    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-bcvl8\""
Sep 09 12:26:32.116302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.115617    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 12:26:32.116415 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.115700    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 12:26:32.116415 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.115760    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 12:26:32.118345 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.118317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"]
Sep 09 12:26:32.164888 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.164850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.164888 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.164890    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.165096 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.164913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8da005ca-91cf-4a00-a21b-7c1291a79f29-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.165096 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.164997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6tns\" (UniqueName: \"kubernetes.io/projected/8da005ca-91cf-4a00-a21b-7c1291a79f29-kube-api-access-q6tns\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.165096 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.165050    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.165194 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.165131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.165194 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.165169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.165258 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.165226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.165291 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.165255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.265892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.265800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6tns\" (UniqueName: \"kubernetes.io/projected/8da005ca-91cf-4a00-a21b-7c1291a79f29-kube-api-access-q6tns\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.265892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.265848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.265892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.265868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.265892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.265884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266253 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.265909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266253 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.265932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266253 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.265967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266253 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.265991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266253 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.266014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8da005ca-91cf-4a00-a21b-7c1291a79f29-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266512 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.266316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266512 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.266427    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266512 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.266493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266711 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.266520    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.266907 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.266886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8da005ca-91cf-4a00-a21b-7c1291a79f29-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.268792 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.268772    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.269122 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.269096    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.274788 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.274755    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8da005ca-91cf-4a00-a21b-7c1291a79f29-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.274917 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.274887    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6tns\" (UniqueName: \"kubernetes.io/projected/8da005ca-91cf-4a00-a21b-7c1291a79f29-kube-api-access-q6tns\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799czcjgf\" (UID: \"8da005ca-91cf-4a00-a21b-7c1291a79f29\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.307465 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.307422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:32.409650 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.409594    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:32.425250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.425215    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:32.700124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:32.700098    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"]
Sep 09 12:26:32.701954 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:26:32.701919    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8da005ca_91cf_4a00_a21b_7c1291a79f29.slice/crio-3239be7df9c0672415f886cba1bf79ce76cb28a98e27e431f85b3f05b581e1dd WatchSource:0}: Error finding container 3239be7df9c0672415f886cba1bf79ce76cb28a98e27e431f85b3f05b581e1dd: Status 404 returned error can't find the container with id 3239be7df9c0672415f886cba1bf79ce76cb28a98e27e431f85b3f05b581e1dd
Sep 09 12:26:33.585819 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:33.585782    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf" event={"ID":"8da005ca-91cf-4a00-a21b-7c1291a79f29","Type":"ContainerStarted","Data":"3239be7df9c0672415f886cba1bf79ce76cb28a98e27e431f85b3f05b581e1dd"}
Sep 09 12:26:35.249131 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:35.249092    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:26:35.249376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:35.249175    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:26:35.249376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:35.249211    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:26:35.596294 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:35.596206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf" event={"ID":"8da005ca-91cf-4a00-a21b-7c1291a79f29","Type":"ContainerStarted","Data":"32a270a328ba03b0f4c9a59eb95226b9109cbed3a5ec01809c26d291d1c9f315"}
Sep 09 12:26:35.617116 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:35.617063    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf" podStartSLOduration=1.073151902 podStartE2EDuration="3.617047137s" totalImagesPullingTime="2.543895235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:35.615543647 +0000 UTC m=+411.939764177" watchObservedRunningTime="2026-09-09 12:26:35.617047137 +0000 UTC m=+411.941267667"
Sep 09 12:26:36.344898 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.344856    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j25dc"]
Sep 09 12:26:36.348032 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.348015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:36.350103 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.350080    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 09 12:26:36.350191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.350158    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lgk7j\""
Sep 09 12:26:36.357055 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.357028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j25dc"]
Sep 09 12:26:36.394594 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.394560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd6v9\" (UniqueName: \"kubernetes.io/projected/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-kube-api-access-qd6v9\") pod \"odh-model-controller-7998c67dc4-j25dc\" (UID: \"e33f9672-6676-4ee2-a0c8-7d0185baeeb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:36.394765 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.394603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert\") pod \"odh-model-controller-7998c67dc4-j25dc\" (UID: \"e33f9672-6676-4ee2-a0c8-7d0185baeeb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:36.418764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.418728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:36.425580 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.425552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:36.430717 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.430696    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:36.477486 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.477451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:36.495394 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.495369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd6v9\" (UniqueName: \"kubernetes.io/projected/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-kube-api-access-qd6v9\") pod \"odh-model-controller-7998c67dc4-j25dc\" (UID: \"e33f9672-6676-4ee2-a0c8-7d0185baeeb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:36.495545 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.495407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert\") pod \"odh-model-controller-7998c67dc4-j25dc\" (UID: \"e33f9672-6676-4ee2-a0c8-7d0185baeeb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:36.495545 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:26:36.495526    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 09 12:26:36.495671 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:26:36.495579    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert podName:e33f9672-6676-4ee2-a0c8-7d0185baeeb2 nodeName:}" failed. No retries permitted until 2026-09-09 12:26:36.995563006 +0000 UTC m=+413.319783515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert") pod "odh-model-controller-7998c67dc4-j25dc" (UID: "e33f9672-6676-4ee2-a0c8-7d0185baeeb2") : secret "odh-model-controller-webhook-cert" not found
Sep 09 12:26:36.508116 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.508076    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd6v9\" (UniqueName: \"kubernetes.io/projected/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-kube-api-access-qd6v9\") pod \"odh-model-controller-7998c67dc4-j25dc\" (UID: \"e33f9672-6676-4ee2-a0c8-7d0185baeeb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:36.542933 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.542891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:36.600671 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.600565    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:36.601574 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.601545    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799czcjgf"
Sep 09 12:26:36.948667 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.948626    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kxrbl"]
Sep 09 12:26:36.951864 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.951841    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:36.952392 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.952370    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:36.953866 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.953847    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 09 12:26:36.953963 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.953867    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-rfkkn\""
Sep 09 12:26:36.962830 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.962802    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kxrbl"]
Sep 09 12:26:36.998920 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.998888    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38b74994-df95-4436-8813-1beba187cf21-tls-certs\") pod \"model-serving-api-54d89bd5fc-kxrbl\" (UID: \"38b74994-df95-4436-8813-1beba187cf21\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:36.999095 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.998933    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert\") pod \"odh-model-controller-7998c67dc4-j25dc\" (UID: \"e33f9672-6676-4ee2-a0c8-7d0185baeeb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:36.999095 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:36.999005    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b9cc\" (UniqueName: \"kubernetes.io/projected/38b74994-df95-4436-8813-1beba187cf21-kube-api-access-6b9cc\") pod \"model-serving-api-54d89bd5fc-kxrbl\" (UID: \"38b74994-df95-4436-8813-1beba187cf21\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:36.999095 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:26:36.999030    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 09 12:26:36.999095 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:26:36.999086    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert podName:e33f9672-6676-4ee2-a0c8-7d0185baeeb2 nodeName:}" failed. No retries permitted until 2026-09-09 12:26:37.999070594 +0000 UTC m=+414.323291103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert") pod "odh-model-controller-7998c67dc4-j25dc" (UID: "e33f9672-6676-4ee2-a0c8-7d0185baeeb2") : secret "odh-model-controller-webhook-cert" not found
Sep 09 12:26:37.100085 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:37.100042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b9cc\" (UniqueName: \"kubernetes.io/projected/38b74994-df95-4436-8813-1beba187cf21-kube-api-access-6b9cc\") pod \"model-serving-api-54d89bd5fc-kxrbl\" (UID: \"38b74994-df95-4436-8813-1beba187cf21\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:37.100264 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:37.100124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38b74994-df95-4436-8813-1beba187cf21-tls-certs\") pod \"model-serving-api-54d89bd5fc-kxrbl\" (UID: \"38b74994-df95-4436-8813-1beba187cf21\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:37.102584 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:37.102554    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38b74994-df95-4436-8813-1beba187cf21-tls-certs\") pod \"model-serving-api-54d89bd5fc-kxrbl\" (UID: \"38b74994-df95-4436-8813-1beba187cf21\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:37.107909 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:37.107885    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b9cc\" (UniqueName: \"kubernetes.io/projected/38b74994-df95-4436-8813-1beba187cf21-kube-api-access-6b9cc\") pod \"model-serving-api-54d89bd5fc-kxrbl\" (UID: \"38b74994-df95-4436-8813-1beba187cf21\") " pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:37.262200 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:37.262102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:37.532172 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:37.532090    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kxrbl"]
Sep 09 12:26:37.535265 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:26:37.535232    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b74994_df95_4436_8813_1beba187cf21.slice/crio-08d8539d1d82c0de83d8d52963828c83bcdd729d9cc8b675d2508925baa83fb3 WatchSource:0}: Error finding container 08d8539d1d82c0de83d8d52963828c83bcdd729d9cc8b675d2508925baa83fb3: Status 404 returned error can't find the container with id 08d8539d1d82c0de83d8d52963828c83bcdd729d9cc8b675d2508925baa83fb3
Sep 09 12:26:37.605581 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:37.605547    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl" event={"ID":"38b74994-df95-4436-8813-1beba187cf21","Type":"ContainerStarted","Data":"08d8539d1d82c0de83d8d52963828c83bcdd729d9cc8b675d2508925baa83fb3"}
Sep 09 12:26:38.005809 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:38.005770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert\") pod \"odh-model-controller-7998c67dc4-j25dc\" (UID: \"e33f9672-6676-4ee2-a0c8-7d0185baeeb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:38.008151 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:38.008120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e33f9672-6676-4ee2-a0c8-7d0185baeeb2-cert\") pod \"odh-model-controller-7998c67dc4-j25dc\" (UID: \"e33f9672-6676-4ee2-a0c8-7d0185baeeb2\") " pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:38.159080 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:38.159037    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:38.197735 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:38.197648    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:38.253455 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:38.253414    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:38.437217 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:38.437135    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j25dc"]
Sep 09 12:26:38.440124 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:26:38.440091    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode33f9672_6676_4ee2_a0c8_7d0185baeeb2.slice/crio-8b87662d53fa782b52c9c672ff91ffb94ad87f6c2be6548467e8504a9af98710 WatchSource:0}: Error finding container 8b87662d53fa782b52c9c672ff91ffb94ad87f6c2be6548467e8504a9af98710: Status 404 returned error can't find the container with id 8b87662d53fa782b52c9c672ff91ffb94ad87f6c2be6548467e8504a9af98710
Sep 09 12:26:38.612199 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:38.612120    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j25dc" event={"ID":"e33f9672-6676-4ee2-a0c8-7d0185baeeb2","Type":"ContainerStarted","Data":"8b87662d53fa782b52c9c672ff91ffb94ad87f6c2be6548467e8504a9af98710"}
Sep 09 12:26:40.622323 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:40.622280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl" event={"ID":"38b74994-df95-4436-8813-1beba187cf21","Type":"ContainerStarted","Data":"4a7c36bcd825bb33b644986892b59d80f2073234ff8df4038285130e83369da4"}
Sep 09 12:26:40.622850 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:40.622366    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:40.639730 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:40.639668    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl" podStartSLOduration=2.134691658 podStartE2EDuration="4.639630683s" totalImagesPullingTime="2.504939025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:40.637765081 +0000 UTC m=+416.961985612" watchObservedRunningTime="2026-09-09 12:26:40.639630683 +0000 UTC m=+416.963851216"
Sep 09 12:26:42.591885 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.591846    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"]
Sep 09 12:26:42.595064 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.595048    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.597431 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.597400    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 12:26:42.597431 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.597413    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 12:26:42.597431 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.597428    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bswxj\""
Sep 09 12:26:42.597689 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.597402    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 12:26:42.607087 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.607062    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"]
Sep 09 12:26:42.631790 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.631750    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j25dc" event={"ID":"e33f9672-6676-4ee2-a0c8-7d0185baeeb2","Type":"ContainerStarted","Data":"110f3a36f7dd1e1bbddb457b8be3ed00f777de34e857200fd792617481ad0062"}
Sep 09 12:26:42.631964 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.631873    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:42.648164 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.648110    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.648164 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.648157    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f56633f-41e1-4539-af32-8462cf7576de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.648408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.648191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.648408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.648218    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f56633f-41e1-4539-af32-8462cf7576de-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.648408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.648241    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.648408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.648262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f56633f-41e1-4539-af32-8462cf7576de-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.648408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.648365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n8mh\" (UniqueName: \"kubernetes.io/projected/3f56633f-41e1-4539-af32-8462cf7576de-kube-api-access-6n8mh\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.659386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.659331    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-j25dc" podStartSLOduration=3.044320762 podStartE2EDuration="6.659313848s" totalImagesPullingTime="3.614993086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:42.657381406 +0000 UTC m=+418.981601938" watchObservedRunningTime="2026-09-09 12:26:42.659313848 +0000 UTC m=+418.983534391"
Sep 09 12:26:42.748895 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.748859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.748895 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.748898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f56633f-41e1-4539-af32-8462cf7576de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.749160 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.748926    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.749160 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.748951    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f56633f-41e1-4539-af32-8462cf7576de-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.749160 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.748974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.749160 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.749001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f56633f-41e1-4539-af32-8462cf7576de-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.749160 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.749065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n8mh\" (UniqueName: \"kubernetes.io/projected/3f56633f-41e1-4539-af32-8462cf7576de-kube-api-access-6n8mh\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.749799 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.749770    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f56633f-41e1-4539-af32-8462cf7576de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.751458 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.751431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f56633f-41e1-4539-af32-8462cf7576de-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.751609 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.751592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.751773 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.751754    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.751904 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.751882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f56633f-41e1-4539-af32-8462cf7576de-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.759010 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.758985    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f56633f-41e1-4539-af32-8462cf7576de-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.759214 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.759191    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n8mh\" (UniqueName: \"kubernetes.io/projected/3f56633f-41e1-4539-af32-8462cf7576de-kube-api-access-6n8mh\") pod \"istiod-openshift-gateway-5677ccb49b-n9cd4\" (UID: \"3f56633f-41e1-4539-af32-8462cf7576de\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:42.903575 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:42.903488    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:43.196249 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:43.196172    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"]
Sep 09 12:26:43.199360 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:26:43.199334    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f56633f_41e1_4539_af32_8462cf7576de.slice/crio-11e9c7eec109e2d6601fd4d494bbb08538e979424f1087da12fd4835362a989d WatchSource:0}: Error finding container 11e9c7eec109e2d6601fd4d494bbb08538e979424f1087da12fd4835362a989d: Status 404 returned error can't find the container with id 11e9c7eec109e2d6601fd4d494bbb08538e979424f1087da12fd4835362a989d
Sep 09 12:26:43.637014 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:43.636977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4" event={"ID":"3f56633f-41e1-4539-af32-8462cf7576de","Type":"ContainerStarted","Data":"11e9c7eec109e2d6601fd4d494bbb08538e979424f1087da12fd4835362a989d"}
Sep 09 12:26:45.332603 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:45.332552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:45.432498 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:45.432448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:45.545842 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:45.545804    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:45.597083 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:45.597051    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:26:45.597190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:45.597113    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:26:45.738323 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:45.738282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:46.233002 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:46.232959    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:46.328101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:46.328062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:46.431110 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:46.431074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:46.650360 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:46.650270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4" event={"ID":"3f56633f-41e1-4539-af32-8462cf7576de","Type":"ContainerStarted","Data":"7853b95d5719305d74e5fcde30416f81baf457fb3f48461de86a3781ad9d8301"}
Sep 09 12:26:46.650508 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:46.650406    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:46.652130 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:46.652109    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4"
Sep 09 12:26:46.674896 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:46.674839    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-n9cd4" podStartSLOduration=2.279971837 podStartE2EDuration="4.674817908s" totalImagesPullingTime="2.394846071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:26:46.674104954 +0000 UTC m=+422.998325484" watchObservedRunningTime="2026-09-09 12:26:46.674817908 +0000 UTC m=+422.999038433"
Sep 09 12:26:51.629998 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:51.629958    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-kxrbl"
Sep 09 12:26:53.639658 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:53.639602    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-j25dc"
Sep 09 12:26:54.515472 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:54.515435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:26:54.576871 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:26:54.576834    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:37.575854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:37.575817    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:37.675198 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:37.675162    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:50.639680 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:50.639621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:51.433764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:51.433733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:52.231225 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:52.231194    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:54.468691 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:54.468652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:54.568176 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:54.568131    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:55.850800 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:55.850766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:56.900998 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:56.900965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:58.042344 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:58.042306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:58.268902 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:58.268868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:27:59.835246 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:27:59.835208    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:00.240022 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:00.239979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:00.471785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:00.471750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:00.568307 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:00.568217    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:02.780223 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:02.780184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:02.882130 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:02.882057    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:03.062131 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:03.062038    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:03.133022 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:03.132988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:05.462409 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:05.462375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:06.481255 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:06.481210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:06.585150 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:06.585109    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:13.148407 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:13.148366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:13.253001 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:13.252967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:17.080150 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:17.080097    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:17.180135 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:17.180101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:21.854455 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:21.854412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:21.954496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:21.954433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:27.091672 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:27.091621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:27.189628 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:27.189591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:27.325628 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:27.325589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:27.417153 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:27.417054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:27.523480 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:27.523434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:27.625891 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:27.625848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:27.717366 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:27.717331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:27.820263 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:27.820228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:34.131051 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:34.130973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:34.231362 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:34.231318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:38.869121 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:38.869017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:38.972778 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:38.972731    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:42.695523 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:42.695486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:42.916316 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:42.916279    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:43.672571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:43.672540    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:43.732908 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:43.732876    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:48.714939 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:48.714901    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:56.266858 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:56.266823    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:56.346096 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:56.346061    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:56.415346 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:56.415312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:57.124417 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:57.124382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:57.230190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:57.230148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:58.975981 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:58.975942    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:28:59.040612 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:28:59.040580    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:12.034655 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:12.034603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:12.089764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:12.089730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:13.928815 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:13.928772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:14.122905 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:14.122870    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:14.438164 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:14.438126    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:26.978354 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:26.978306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:27.078238 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:27.078189    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:33.980087 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:33.980052    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:34.074554 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:34.074515    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:34.146788 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:34.146748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:37.474926 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:37.474869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:37.575301 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:37.575267    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:29:46.328970 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:29:46.328930    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:10.128092 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:10.128013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:10.228322 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:10.228282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:11.523149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:11.523116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:11.630612 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:11.630561    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:11.735169 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:11.735130    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:11.836193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:11.836098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:12.073941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.073904    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:12.172943 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.172853    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:12.248424 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.248389    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"]
Sep 09 12:30:12.251484 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.251461    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:12.253813 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.253780    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 12:30:12.253813 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.253780    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-c8fzl\""
Sep 09 12:30:12.261716 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.261695    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"]
Sep 09 12:30:12.363872 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.363834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c881f1f-f3ec-4752-834e-2a694ab52ce2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2zjb7\" (UID: \"5c881f1f-f3ec-4752-834e-2a694ab52ce2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:12.364015 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.363899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7mkg\" (UniqueName: \"kubernetes.io/projected/5c881f1f-f3ec-4752-834e-2a694ab52ce2-kube-api-access-d7mkg\") pod \"payload-pre-processing-59fbc47bf8-2zjb7\" (UID: \"5c881f1f-f3ec-4752-834e-2a694ab52ce2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:12.464592 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.464544    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7mkg\" (UniqueName: \"kubernetes.io/projected/5c881f1f-f3ec-4752-834e-2a694ab52ce2-kube-api-access-d7mkg\") pod \"payload-pre-processing-59fbc47bf8-2zjb7\" (UID: \"5c881f1f-f3ec-4752-834e-2a694ab52ce2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:12.464765 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.464623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c881f1f-f3ec-4752-834e-2a694ab52ce2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2zjb7\" (UID: \"5c881f1f-f3ec-4752-834e-2a694ab52ce2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:12.465293 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.465273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c881f1f-f3ec-4752-834e-2a694ab52ce2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-2zjb7\" (UID: \"5c881f1f-f3ec-4752-834e-2a694ab52ce2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:12.473280 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.473251    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7mkg\" (UniqueName: \"kubernetes.io/projected/5c881f1f-f3ec-4752-834e-2a694ab52ce2-kube-api-access-d7mkg\") pod \"payload-pre-processing-59fbc47bf8-2zjb7\" (UID: \"5c881f1f-f3ec-4752-834e-2a694ab52ce2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:12.562153 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.562115    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:12.830288 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:12.830206    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"]
Sep 09 12:30:12.833129 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:30:12.833088    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c881f1f_f3ec_4752_834e_2a694ab52ce2.slice/crio-98b255aaba90eb1a2a3d95644779946f957f2cfaff89af07c358c5966fd5cbd1 WatchSource:0}: Error finding container 98b255aaba90eb1a2a3d95644779946f957f2cfaff89af07c358c5966fd5cbd1: Status 404 returned error can't find the container with id 98b255aaba90eb1a2a3d95644779946f957f2cfaff89af07c358c5966fd5cbd1
Sep 09 12:30:13.479507 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:13.479472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7" event={"ID":"5c881f1f-f3ec-4752-834e-2a694ab52ce2","Type":"ContainerStarted","Data":"98b255aaba90eb1a2a3d95644779946f957f2cfaff89af07c358c5966fd5cbd1"}
Sep 09 12:30:15.490452 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:15.490416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7" event={"ID":"5c881f1f-f3ec-4752-834e-2a694ab52ce2","Type":"ContainerStarted","Data":"d703cd4223cf52a4c969c489912296f8b64bf1c0aadec1b1daae4d291669e3a2"}
Sep 09 12:30:15.490860 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:15.490526    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:15.507986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:15.507943    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7" podStartSLOduration=0.993278114 podStartE2EDuration="3.507929181s" totalImagesPullingTime="2.514651067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:15.507170375 +0000 UTC m=+631.831390919" watchObservedRunningTime="2026-09-09 12:30:15.507929181 +0000 UTC m=+631.832149713"
Sep 09 12:30:21.991139 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:21.991094    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:22.086169 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:22.086138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:26.496775 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:26.496744    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-2zjb7"
Sep 09 12:30:32.335785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.335753    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"]
Sep 09 12:30:32.342620 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.342596    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.345893 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.345867    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2r6ws\""
Sep 09 12:30:32.346012 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.345931    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 12:30:32.346012 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.345867    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 09 12:30:32.346012 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.345874    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 12:30:32.349149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.349124    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"]
Sep 09 12:30:32.411140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.411110    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.411280 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.411143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.411280 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.411165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.411280 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.411246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.411280 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.411269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp4xd\" (UniqueName: \"kubernetes.io/projected/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-kube-api-access-tp4xd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.411461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.411354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.417481 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.417447    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:32.511980 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.511947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.511980 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.511982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.512158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.512003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.512158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.512045    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.512158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.512069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp4xd\" (UniqueName: \"kubernetes.io/projected/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-kube-api-access-tp4xd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.512158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.512115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.512475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.512443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.512475 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.512465    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.512624 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.512524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.514209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.514189    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.514463 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.514445    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.520073 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.520052    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp4xd\" (UniqueName: \"kubernetes.io/projected/5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88-kube-api-access-tp4xd\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt\" (UID: \"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.654734 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.654624    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:32.941548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.941524    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"]
Sep 09 12:30:32.943548 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:30:32.943518    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aced2bc_6ec2_4fe6_96c5_78b60bb6ae88.slice/crio-326d93fc5f544a0216920e36e38bc1c1771a9b3c9803579018fcd933a416f008 WatchSource:0}: Error finding container 326d93fc5f544a0216920e36e38bc1c1771a9b3c9803579018fcd933a416f008: Status 404 returned error can't find the container with id 326d93fc5f544a0216920e36e38bc1c1771a9b3c9803579018fcd933a416f008
Sep 09 12:30:32.944900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:32.944884    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:30:33.561423 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:33.561385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt" event={"ID":"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88","Type":"ContainerStarted","Data":"326d93fc5f544a0216920e36e38bc1c1771a9b3c9803579018fcd933a416f008"}
Sep 09 12:30:34.567686 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:34.567652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt" event={"ID":"5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88","Type":"ContainerStarted","Data":"0f80e0e51baadab4b518b12a5728d84fdefb131ae4d6bdd3a3b5de680bb7b7c4"}
Sep 09 12:30:34.590162 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:34.590087    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt" podStartSLOduration=1.627273942 podStartE2EDuration="2.590067224s" totalImagesPullingTime="962.793282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:34.589334619 +0000 UTC m=+650.913555142" watchObservedRunningTime="2026-09-09 12:30:34.590067224 +0000 UTC m=+650.914287756"
Sep 09 12:30:40.725191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.725158    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"]
Sep 09 12:30:40.728211 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.728195    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.730344 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.730325    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 09 12:30:40.737763 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.737721    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"]
Sep 09 12:30:40.782002 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.781976    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.782109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.782007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b41c861-0b06-4007-b75d-a5769b7f0140-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.782109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.782040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.782109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.782090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.782235 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.782120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.782235 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.782141    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bqwv\" (UniqueName: \"kubernetes.io/projected/7b41c861-0b06-4007-b75d-a5769b7f0140-kube-api-access-2bqwv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.787459 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.787435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:40.883424 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.883424 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.883762 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.883762 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bqwv\" (UniqueName: \"kubernetes.io/projected/7b41c861-0b06-4007-b75d-a5769b7f0140-kube-api-access-2bqwv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.883762 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.883762 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b41c861-0b06-4007-b75d-a5769b7f0140-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.883972 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.883972 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.883972 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.883962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.885814 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.885796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b41c861-0b06-4007-b75d-a5769b7f0140-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.886104 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.886086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b41c861-0b06-4007-b75d-a5769b7f0140-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:40.891460 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:40.891436    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bqwv\" (UniqueName: \"kubernetes.io/projected/7b41c861-0b06-4007-b75d-a5769b7f0140-kube-api-access-2bqwv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vfw9j\" (UID: \"7b41c861-0b06-4007-b75d-a5769b7f0140\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:41.038660 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:41.038552    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:41.311131 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:41.311064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"]
Sep 09 12:30:41.313292 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:30:41.313264    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b41c861_0b06_4007_b75d_a5769b7f0140.slice/crio-a10a8a029a8dd324e1b7b59e0224f009b417ad0ce6905b24b51c201f3a71bb7f WatchSource:0}: Error finding container a10a8a029a8dd324e1b7b59e0224f009b417ad0ce6905b24b51c201f3a71bb7f: Status 404 returned error can't find the container with id a10a8a029a8dd324e1b7b59e0224f009b417ad0ce6905b24b51c201f3a71bb7f
Sep 09 12:30:41.597964 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:41.597930    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j" event={"ID":"7b41c861-0b06-4007-b75d-a5769b7f0140","Type":"ContainerStarted","Data":"088ba398dff239c00fc6034b7b560b0224be43e08592887273c2829d04ee22fb"}
Sep 09 12:30:41.598112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:41.597989    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j" event={"ID":"7b41c861-0b06-4007-b75d-a5769b7f0140","Type":"ContainerStarted","Data":"a10a8a029a8dd324e1b7b59e0224f009b417ad0ce6905b24b51c201f3a71bb7f"}
Sep 09 12:30:41.620003 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:41.616746    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j" podStartSLOduration=1.408403798 podStartE2EDuration="1.616721667s" totalImagesPullingTime="208.317869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:30:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:30:41.614108957 +0000 UTC m=+657.938329499" watchObservedRunningTime="2026-09-09 12:30:41.616721667 +0000 UTC m=+657.940942210"
Sep 09 12:30:42.655502 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:42.655457    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:42.655967 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:42.655525    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:42.667936 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:42.667908    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:43.617954 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:43.617920    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt"
Sep 09 12:30:45.192845 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:45.192812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:30:51.039236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:51.039202    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:51.039618 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:51.039272    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:51.051584 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:51.051561    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:51.649685 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:51.649654    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vfw9j"
Sep 09 12:30:55.118465 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:30:55.118423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:19.744943 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.744912    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"]
Sep 09 12:31:19.746970 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.746954    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.749205 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.749185    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 09 12:31:19.759804 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.759782    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"]
Sep 09 12:31:19.807624 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.807587    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.807624 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.807629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.807847 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.807715    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.807847 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.807755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxml2\" (UniqueName: \"kubernetes.io/projected/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-kube-api-access-pxml2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.807847 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.807783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.807847 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.807804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.903601 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.903544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:19.908549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.908526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.908627 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.908562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxml2\" (UniqueName: \"kubernetes.io/projected/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-kube-api-access-pxml2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.908627 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.908580    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.908627 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.908608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.908790 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.908666    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.908790 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.908695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.909020 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.908999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.909088 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.909029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.909088 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.909076    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.910901 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.910881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.911132 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.911115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:19.916353 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:19.916332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxml2\" (UniqueName: \"kubernetes.io/projected/6565fd7e-8d72-4f3c-b457-46f0d86f76c0-kube-api-access-pxml2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj\" (UID: \"6565fd7e-8d72-4f3c-b457-46f0d86f76c0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:20.057557 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:20.057463    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:20.332616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:20.332591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"]
Sep 09 12:31:20.335232 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:31:20.335200    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6565fd7e_8d72_4f3c_b457_46f0d86f76c0.slice/crio-cca0961855835ffc58cfcb01f438ac238f0c6d1081429e484663d6c547168575 WatchSource:0}: Error finding container cca0961855835ffc58cfcb01f438ac238f0c6d1081429e484663d6c547168575: Status 404 returned error can't find the container with id cca0961855835ffc58cfcb01f438ac238f0c6d1081429e484663d6c547168575
Sep 09 12:31:20.766781 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:20.766750    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj" event={"ID":"6565fd7e-8d72-4f3c-b457-46f0d86f76c0","Type":"ContainerStarted","Data":"7cea90787a350e2f2d8a36c5e036cc5e554ef97d8e72ed099e6d7352811c12e0"}
Sep 09 12:31:20.767183 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:20.766799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj" event={"ID":"6565fd7e-8d72-4f3c-b457-46f0d86f76c0","Type":"ContainerStarted","Data":"cca0961855835ffc58cfcb01f438ac238f0c6d1081429e484663d6c547168575"}
Sep 09 12:31:20.793199 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:20.793141    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj" podStartSLOduration=1.594229562 podStartE2EDuration="1.793125289s" totalImagesPullingTime="198.895727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:31:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:31:20.790968235 +0000 UTC m=+697.115188777" watchObservedRunningTime="2026-09-09 12:31:20.793125289 +0000 UTC m=+697.117345820"
Sep 09 12:31:29.378606 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:29.378566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:30.058213 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:30.058171    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:30.058401 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:30.058276    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:30.070496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:30.070470    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:30.818519 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:30.818492    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj"
Sep 09 12:31:33.147012 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:33.146975    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:33.248225 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:33.248190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:31:43.795380 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:31:43.795342    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:32:41.150380 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:32:41.150340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:32:41.224100 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:32:41.224069    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:32:48.410757 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:32:48.410710    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:32:53.452603 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:32:53.452570    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:32:53.544973 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:32:53.544943    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:32:54.673023 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:32:54.672987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:32:54.770957 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:32:54.770921    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:10.579206 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:10.579113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:10.681178 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:10.681137    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:10.903018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:10.902933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:11.215075 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:11.215045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:11.960997 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:11.960962    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:12.053545 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:12.053513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:12.577140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:12.577099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:13.948910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:13.948871    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:14.057358 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:14.057325    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:15.203344 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.203307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:15.299243 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.299209    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:15.790249 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.790217    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:15.894023 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.893988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:15.895303 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.895281    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"]
Sep 09 12:33:15.898384 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.898364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:15.900821 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.900800    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-1e223f-dockercfg-j4ncw\""
Sep 09 12:33:15.900921 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.900822    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-1e223f\""
Sep 09 12:33:15.908826 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.908802    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"]
Sep 09 12:33:15.945150 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.945118    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"]
Sep 09 12:33:15.948330 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.948309    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:15.958197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.958174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"]
Sep 09 12:33:15.962539 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.962517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxdwc\" (UniqueName: \"kubernetes.io/projected/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-kube-api-access-dxdwc\") pod \"payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl\" (UID: \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:15.962664 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.962556    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzfnp\" (UniqueName: \"kubernetes.io/projected/641f1344-1baa-4b01-b74b-9f9f45c26d9c-kube-api-access-vzfnp\") pod \"payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c\" (UID: \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:15.962664 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.962576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/641f1344-1baa-4b01-b74b-9f9f45c26d9c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c\" (UID: \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:15.962782 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:15.962718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl\" (UID: \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:16.063219 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.063135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxdwc\" (UniqueName: \"kubernetes.io/projected/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-kube-api-access-dxdwc\") pod \"payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl\" (UID: \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:16.063219 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.063208    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzfnp\" (UniqueName: \"kubernetes.io/projected/641f1344-1baa-4b01-b74b-9f9f45c26d9c-kube-api-access-vzfnp\") pod \"payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c\" (UID: \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:16.063506 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.063242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/641f1344-1baa-4b01-b74b-9f9f45c26d9c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c\" (UID: \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:16.063506 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.063295    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl\" (UID: \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:16.064024 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.064004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/641f1344-1baa-4b01-b74b-9f9f45c26d9c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c\" (UID: \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:16.064164 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.064144    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl\" (UID: \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:16.071502 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.071470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxdwc\" (UniqueName: \"kubernetes.io/projected/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-kube-api-access-dxdwc\") pod \"payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl\" (UID: \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:16.071744 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.071720    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzfnp\" (UniqueName: \"kubernetes.io/projected/641f1344-1baa-4b01-b74b-9f9f45c26d9c-kube-api-access-vzfnp\") pod \"payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c\" (UID: \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:16.209099 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.209057    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:16.258749 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.258551    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:16.477582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.477546    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"]
Sep 09 12:33:16.481134 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:33:16.481107    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641f1344_1baa_4b01_b74b_9f9f45c26d9c.slice/crio-6debf013544c83cc30e38624bfcad7c7088956acc4b1a07895710f433ac29652 WatchSource:0}: Error finding container 6debf013544c83cc30e38624bfcad7c7088956acc4b1a07895710f433ac29652: Status 404 returned error can't find the container with id 6debf013544c83cc30e38624bfcad7c7088956acc4b1a07895710f433ac29652
Sep 09 12:33:16.537565 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:16.537535    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"]
Sep 09 12:33:16.539340 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:33:16.539316    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod712bd04a_7cab_4bc2_a4b8_4edcf4d9e1b6.slice/crio-739dca7ea6d0fa24b24bdcc6615635631d2f1f3884c12e99e7578c5c9a0097db WatchSource:0}: Error finding container 739dca7ea6d0fa24b24bdcc6615635631d2f1f3884c12e99e7578c5c9a0097db: Status 404 returned error can't find the container with id 739dca7ea6d0fa24b24bdcc6615635631d2f1f3884c12e99e7578c5c9a0097db
Sep 09 12:33:17.249190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:17.249151    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c" event={"ID":"641f1344-1baa-4b01-b74b-9f9f45c26d9c","Type":"ContainerStarted","Data":"e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e"}
Sep 09 12:33:17.249693 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:17.249223    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:17.249693 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:17.249248    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c" event={"ID":"641f1344-1baa-4b01-b74b-9f9f45c26d9c","Type":"ContainerStarted","Data":"6debf013544c83cc30e38624bfcad7c7088956acc4b1a07895710f433ac29652"}
Sep 09 12:33:17.250854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:17.250832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl" event={"ID":"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6","Type":"ContainerStarted","Data":"fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2"}
Sep 09 12:33:17.250962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:17.250877    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl" event={"ID":"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6","Type":"ContainerStarted","Data":"739dca7ea6d0fa24b24bdcc6615635631d2f1f3884c12e99e7578c5c9a0097db"}
Sep 09 12:33:17.250962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:17.250910    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:17.265851 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:17.265788    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c" podStartSLOduration=2.265770315 podStartE2EDuration="2.265770315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:17.265568027 +0000 UTC m=+813.589788560" watchObservedRunningTime="2026-09-09 12:33:17.265770315 +0000 UTC m=+813.589990851"
Sep 09 12:33:17.284921 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:17.284869    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl" podStartSLOduration=2.28485205 podStartE2EDuration="2.28485205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:17.282424388 +0000 UTC m=+813.606644919" watchObservedRunningTime="2026-09-09 12:33:17.28485205 +0000 UTC m=+813.609072584"
Sep 09 12:33:19.372070 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.372031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:19.461976 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.461939    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:19.680144 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.680052    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:19.748269 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.748236    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"]
Sep 09 12:33:19.751571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.751549    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.753967 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.753944    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f09b7dee-openshift-default-dockercfg-s5k7x\""
Sep 09 12:33:19.765111 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.765087    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"]
Sep 09 12:33:19.782415 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.782389    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:19.795039 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-podinfo\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.795163 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795052    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-envoy\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.795163 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-token\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.795163 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istiod-ca-cert\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.795335 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-data\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.795335 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795249    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmtrx\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-kube-api-access-jmtrx\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.795335 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-socket\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.795446 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-credential-socket\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.795446 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.795403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-certs\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896073 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-certs\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-podinfo\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-envoy\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-token\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896171    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istiod-ca-cert\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-data\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmtrx\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-kube-api-access-jmtrx\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-socket\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-credential-socket\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896519    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-certs\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896604    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-credential-socket\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896780    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-data\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.896898 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.896860    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-socket\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.897171 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.897147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istiod-ca-cert\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.898812 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.898778    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-podinfo\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.899361 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.899337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-envoy\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.905296 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.905266    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmtrx\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-kube-api-access-jmtrx\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:19.905403 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:19.905383    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-token\") pod \"e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") " pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:20.034306 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:20.034266    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:20.064905 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:20.064872    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:20.135534 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:20.135492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:20.340693 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:20.340668    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"]
Sep 09 12:33:20.342590 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:33:20.342560    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee75ee85_b41a_4b19_9e1c_14d02ed4c36e.slice/crio-65fe1b7bbb36e12ef2eca455222c4d3136a764f9a5dca505504e3df63a1647ed WatchSource:0}: Error finding container 65fe1b7bbb36e12ef2eca455222c4d3136a764f9a5dca505504e3df63a1647ed: Status 404 returned error can't find the container with id 65fe1b7bbb36e12ef2eca455222c4d3136a764f9a5dca505504e3df63a1647ed
Sep 09 12:33:20.345193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:20.345163    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:33:20.345271 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:20.345224    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:33:20.345271 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:20.345254    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:33:21.268945 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:21.268901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" event={"ID":"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e","Type":"ContainerStarted","Data":"7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e"}
Sep 09 12:33:21.269382 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:21.268956    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" event={"ID":"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e","Type":"ContainerStarted","Data":"65fe1b7bbb36e12ef2eca455222c4d3136a764f9a5dca505504e3df63a1647ed"}
Sep 09 12:33:21.291466 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:21.291418    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" podStartSLOduration=2.291401916 podStartE2EDuration="2.291401916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:21.289601052 +0000 UTC m=+817.613821598" watchObservedRunningTime="2026-09-09 12:33:21.291401916 +0000 UTC m=+817.615622447"
Sep 09 12:33:22.066013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:22.065976    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:22.070785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:22.070758    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:22.273702 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:22.273668    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:22.274802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:22.274776    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:22.755621 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:22.755582    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:22.887195 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:22.887152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:23.003384 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:23.003347    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:23.106172 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:23.106081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:24.066428 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:24.066397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:24.151225 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:24.151183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:24.675698 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:24.675661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:24.774704 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:24.774665    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:25.324437 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.324407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:25.403322 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.403287    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:25.475274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.475227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:25.838743 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.838710    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:25.916582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.916547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:25.940915 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.940877    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"]
Sep 09 12:33:25.944766 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.944743    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:25.946722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.946703    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-e917b0\""
Sep 09 12:33:25.952109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.952087    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"]
Sep 09 12:33:25.966747 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.966721    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"]
Sep 09 12:33:25.969864 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.969848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:25.971945 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.971925    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-e917b0-dockercfg-f65g7\""
Sep 09 12:33:25.978599 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:25.978578    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"]
Sep 09 12:33:26.050710 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.050675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h97nk\" (UniqueName: \"kubernetes.io/projected/41f64af0-57f8-4275-bd84-93815ffef074-kube-api-access-h97nk\") pod \"payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc\" (UID: \"41f64af0-57f8-4275-bd84-93815ffef074\") " pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:26.050861 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.050743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrlbv\" (UniqueName: \"kubernetes.io/projected/45385990-76d4-41dc-b6fc-28b95c74702a-kube-api-access-zrlbv\") pod \"payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp\" (UID: \"45385990-76d4-41dc-b6fc-28b95c74702a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:26.050861 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.050787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45385990-76d4-41dc-b6fc-28b95c74702a-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp\" (UID: \"45385990-76d4-41dc-b6fc-28b95c74702a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:26.050932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.050866    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f64af0-57f8-4275-bd84-93815ffef074-plugins-config-volume\") pod \"payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc\" (UID: \"41f64af0-57f8-4275-bd84-93815ffef074\") " pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:26.151544 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.151472    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f64af0-57f8-4275-bd84-93815ffef074-plugins-config-volume\") pod \"payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc\" (UID: \"41f64af0-57f8-4275-bd84-93815ffef074\") " pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:26.151544 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.151532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h97nk\" (UniqueName: \"kubernetes.io/projected/41f64af0-57f8-4275-bd84-93815ffef074-kube-api-access-h97nk\") pod \"payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc\" (UID: \"41f64af0-57f8-4275-bd84-93815ffef074\") " pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:26.151763 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.151555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrlbv\" (UniqueName: \"kubernetes.io/projected/45385990-76d4-41dc-b6fc-28b95c74702a-kube-api-access-zrlbv\") pod \"payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp\" (UID: \"45385990-76d4-41dc-b6fc-28b95c74702a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:26.151763 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.151575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45385990-76d4-41dc-b6fc-28b95c74702a-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp\" (UID: \"45385990-76d4-41dc-b6fc-28b95c74702a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:26.152264 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.152243    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f64af0-57f8-4275-bd84-93815ffef074-plugins-config-volume\") pod \"payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc\" (UID: \"41f64af0-57f8-4275-bd84-93815ffef074\") " pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:26.152302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.152260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45385990-76d4-41dc-b6fc-28b95c74702a-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp\" (UID: \"45385990-76d4-41dc-b6fc-28b95c74702a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:26.159896 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.159869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h97nk\" (UniqueName: \"kubernetes.io/projected/41f64af0-57f8-4275-bd84-93815ffef074-kube-api-access-h97nk\") pod \"payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc\" (UID: \"41f64af0-57f8-4275-bd84-93815ffef074\") " pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:26.160102 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.160082    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrlbv\" (UniqueName: \"kubernetes.io/projected/45385990-76d4-41dc-b6fc-28b95c74702a-kube-api-access-zrlbv\") pod \"payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp\" (UID: \"45385990-76d4-41dc-b6fc-28b95c74702a\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:26.255631 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.255599    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:26.281839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.281802    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:26.307873 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.307842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:26.530605 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.530582    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"]
Sep 09 12:33:26.532317 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:33:26.532287    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45385990_76d4_41dc_b6fc_28b95c74702a.slice/crio-74eae468d689aa0ad80aa2c6eb0306653fd8f214cae82d4c16d2dbfbfb0813dc WatchSource:0}: Error finding container 74eae468d689aa0ad80aa2c6eb0306653fd8f214cae82d4c16d2dbfbfb0813dc: Status 404 returned error can't find the container with id 74eae468d689aa0ad80aa2c6eb0306653fd8f214cae82d4c16d2dbfbfb0813dc
Sep 09 12:33:26.558785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:26.558762    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"]
Sep 09 12:33:26.570745 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:33:26.570718    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f64af0_57f8_4275_bd84_93815ffef074.slice/crio-2ede9af90971c7ede3c462f1d0c5b935daae5bad2d604ff21d0c380d32c9e572 WatchSource:0}: Error finding container 2ede9af90971c7ede3c462f1d0c5b935daae5bad2d604ff21d0c380d32c9e572: Status 404 returned error can't find the container with id 2ede9af90971c7ede3c462f1d0c5b935daae5bad2d604ff21d0c380d32c9e572
Sep 09 12:33:27.174207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.174172    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:27.298010 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.297969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc" event={"ID":"41f64af0-57f8-4275-bd84-93815ffef074","Type":"ContainerStarted","Data":"2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c"}
Sep 09 12:33:27.298202 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.298097    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc" event={"ID":"41f64af0-57f8-4275-bd84-93815ffef074","Type":"ContainerStarted","Data":"2ede9af90971c7ede3c462f1d0c5b935daae5bad2d604ff21d0c380d32c9e572"}
Sep 09 12:33:27.298202 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.298131    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:27.299552 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.299529    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp" event={"ID":"45385990-76d4-41dc-b6fc-28b95c74702a","Type":"ContainerStarted","Data":"fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648"}
Sep 09 12:33:27.299660 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.299567    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp" event={"ID":"45385990-76d4-41dc-b6fc-28b95c74702a","Type":"ContainerStarted","Data":"74eae468d689aa0ad80aa2c6eb0306653fd8f214cae82d4c16d2dbfbfb0813dc"}
Sep 09 12:33:27.299660 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.299616    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:27.315255 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.315206    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc" podStartSLOduration=2.315189794 podStartE2EDuration="2.315189794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:27.313934187 +0000 UTC m=+823.638154722" watchObservedRunningTime="2026-09-09 12:33:27.315189794 +0000 UTC m=+823.639410325"
Sep 09 12:33:27.328317 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:27.328264    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp" podStartSLOduration=2.328246988 podStartE2EDuration="2.328246988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:27.327687025 +0000 UTC m=+823.651907555" watchObservedRunningTime="2026-09-09 12:33:27.328246988 +0000 UTC m=+823.652467519"
Sep 09 12:33:28.256705 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:28.256674    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:33:28.257108 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:28.256746    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:33:29.404265 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:29.404229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:29.548197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:29.548153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:29.642715 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:29.642670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:29.747089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:29.747039    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:32.334209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:32.334173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:32.435547 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:32.435509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:34.183357 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:34.183321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:35.966483 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:35.966430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:36.067563 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:36.067527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:36.194151 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:36.194113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:36.298261 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:36.298166    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:38.307625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:38.307594    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:33:38.308739 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:38.307689    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:33:38.315775 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:38.315747    2579 patch_prober.go:28] interesting pod/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 12:33:38.316095 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:38.315796    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 12:33:38.825926 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:38.825884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:39.148188 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.148099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:39.359404 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.359371    2579 generic.go:356] "Generic (PLEG): container finished" podID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerID="7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e" exitCode=0
Sep 09 12:33:39.359776 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.359446    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" event={"ID":"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e","Type":"ContainerDied","Data":"7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e"}
Sep 09 12:33:39.359848 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.359834    2579 scope.go:117] "RemoveContainer" containerID="7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e"
Sep 09 12:33:39.360886 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.360861    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:33:39.360962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.360928    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:33:39.360998 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.360961    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:33:39.450062 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.450027    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:39.629600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:39.629554    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:40.373585 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:40.373544    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" event={"ID":"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e","Type":"ContainerStarted","Data":"4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317"}
Sep 09 12:33:41.066252 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:41.066218    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:41.066554 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:41.066254    2579 patch_prober.go:28] interesting pod/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 09 12:33:41.066772 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:41.066746    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 09 12:33:42.066268 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:42.066226    2579 patch_prober.go:28] interesting pod/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 09 12:33:42.067292 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:42.066884    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 09 12:33:42.152747 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:42.152710    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:42.281945 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:42.281902    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:42.875975 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:42.875938    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:42.980845 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:42.980808    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:43.066090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:43.066049    2579 patch_prober.go:28] interesting pod/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 09 12:33:43.066381 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:43.066350    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 09 12:33:43.076784 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:43.076744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:43.275701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:43.275662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:43.379155 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:43.379096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:43.477005 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:43.476968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:43.982877 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:43.982846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:44.068661 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.068181    2579 patch_prober.go:28] interesting pod/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 09 12:33:44.068661 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.068252    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 09 12:33:44.081409 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.081377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:44.111398 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.111371    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"]
Sep 09 12:33:44.114477 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.114462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:44.116630 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.116611    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5e5923\""
Sep 09 12:33:44.116882 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.116857    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-5e5923-dockercfg-8jnb9\""
Sep 09 12:33:44.125247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.125227    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"]
Sep 09 12:33:44.172438 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.169014    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"]
Sep 09 12:33:44.175198 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.175172    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:44.175431 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.175277    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"]
Sep 09 12:33:44.219098 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.219059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1801864-6611-4730-bb61-13931c0fbcc0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv\" (UID: \"a1801864-6611-4730-bb61-13931c0fbcc0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:44.219098 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.219100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt6mm\" (UniqueName: \"kubernetes.io/projected/a1801864-6611-4730-bb61-13931c0fbcc0-kube-api-access-jt6mm\") pod \"payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv\" (UID: \"a1801864-6611-4730-bb61-13931c0fbcc0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:44.319658 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.319549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1801864-6611-4730-bb61-13931c0fbcc0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv\" (UID: \"a1801864-6611-4730-bb61-13931c0fbcc0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:44.319658 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.319595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt6mm\" (UniqueName: \"kubernetes.io/projected/a1801864-6611-4730-bb61-13931c0fbcc0-kube-api-access-jt6mm\") pod \"payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv\" (UID: \"a1801864-6611-4730-bb61-13931c0fbcc0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:44.319870 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.319669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6\" (UID: \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:44.319870 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.319706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtxgk\" (UniqueName: \"kubernetes.io/projected/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-kube-api-access-rtxgk\") pod \"payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6\" (UID: \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:44.320274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.320254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1801864-6611-4730-bb61-13931c0fbcc0-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv\" (UID: \"a1801864-6611-4730-bb61-13931c0fbcc0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:44.327836 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.327808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt6mm\" (UniqueName: \"kubernetes.io/projected/a1801864-6611-4730-bb61-13931c0fbcc0-kube-api-access-jt6mm\") pod \"payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv\" (UID: \"a1801864-6611-4730-bb61-13931c0fbcc0\") " pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:44.374274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.374238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:44.420124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.420092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6\" (UID: \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:44.420278 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.420130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtxgk\" (UniqueName: \"kubernetes.io/projected/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-kube-api-access-rtxgk\") pod \"payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6\" (UID: \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:44.420828 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.420809    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6\" (UID: \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:44.424017 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.423997    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:44.429630 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.429600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtxgk\" (UniqueName: \"kubernetes.io/projected/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-kube-api-access-rtxgk\") pod \"payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6\" (UID: \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:44.485794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.485756    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:44.716228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.716198    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"]
Sep 09 12:33:44.718589 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:33:44.718559    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1801864_6611_4730_bb61_13931c0fbcc0.slice/crio-753679d8c5fff648f64112bb2abb9149c1c0be8a51a93aa61b75d102ae37ac4c WatchSource:0}: Error finding container 753679d8c5fff648f64112bb2abb9149c1c0be8a51a93aa61b75d102ae37ac4c: Status 404 returned error can't find the container with id 753679d8c5fff648f64112bb2abb9149c1c0be8a51a93aa61b75d102ae37ac4c
Sep 09 12:33:44.774686 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:44.774653    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"]
Sep 09 12:33:44.779756 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:33:44.779730    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6665a5d_8e94_4a83_a6f0_fb4c4616e1e0.slice/crio-d3ad33a690c6217df85b4cc01f5feb9209d962e9217223454083996256e4e8a9 WatchSource:0}: Error finding container d3ad33a690c6217df85b4cc01f5feb9209d962e9217223454083996256e4e8a9: Status 404 returned error can't find the container with id d3ad33a690c6217df85b4cc01f5feb9209d962e9217223454083996256e4e8a9
Sep 09 12:33:45.066248 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.066142    2579 patch_prober.go:28] interesting pod/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 09 12:33:45.066248 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.066220    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 09 12:33:45.399335 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.399231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6" event={"ID":"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0","Type":"ContainerStarted","Data":"28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6"}
Sep 09 12:33:45.399335 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.399298    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:45.399335 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.399323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6" event={"ID":"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0","Type":"ContainerStarted","Data":"d3ad33a690c6217df85b4cc01f5feb9209d962e9217223454083996256e4e8a9"}
Sep 09 12:33:45.400800 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.400776    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv" event={"ID":"a1801864-6611-4730-bb61-13931c0fbcc0","Type":"ContainerStarted","Data":"0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34"}
Sep 09 12:33:45.400906 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.400822    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv" event={"ID":"a1801864-6611-4730-bb61-13931c0fbcc0","Type":"ContainerStarted","Data":"753679d8c5fff648f64112bb2abb9149c1c0be8a51a93aa61b75d102ae37ac4c"}
Sep 09 12:33:45.400974 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.400961    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:33:45.414710 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.414554    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6" podStartSLOduration=1.414539639 podStartE2EDuration="1.414539639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:45.414269783 +0000 UTC m=+841.738490314" watchObservedRunningTime="2026-09-09 12:33:45.414539639 +0000 UTC m=+841.738760169"
Sep 09 12:33:45.432308 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.432256    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv" podStartSLOduration=1.432235522 podStartE2EDuration="1.432235522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:33:45.43043221 +0000 UTC m=+841.754652746" watchObservedRunningTime="2026-09-09 12:33:45.432235522 +0000 UTC m=+841.756456058"
Sep 09 12:33:45.727501 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:45.727460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:46.069732 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:46.069648    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:46.069958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:46.069937    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:46.070576 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:46.070560    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:33:46.696355 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:46.696312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:47.556413 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:47.556385    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:48.137223 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.137189    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"]
Sep 09 12:33:48.140764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.140747    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"
Sep 09 12:33:48.142835 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.142809    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:33:48.142966 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.142841    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:33:48.152412 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.152382    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"]
Sep 09 12:33:48.256204 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.256169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928f8bee-de03-4fc3-afd7-2feeabefb6b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq\" (UID: \"928f8bee-de03-4fc3-afd7-2feeabefb6b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"
Sep 09 12:33:48.356706 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.356667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928f8bee-de03-4fc3-afd7-2feeabefb6b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq\" (UID: \"928f8bee-de03-4fc3-afd7-2feeabefb6b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"
Sep 09 12:33:48.357289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.357270    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928f8bee-de03-4fc3-afd7-2feeabefb6b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq\" (UID: \"928f8bee-de03-4fc3-afd7-2feeabefb6b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"
Sep 09 12:33:48.451425 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.451393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"
Sep 09 12:33:48.564293 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.564255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:48.719055 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:48.719025    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"]
Sep 09 12:33:48.721596 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:33:48.721571    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928f8bee_de03_4fc3_afd7_2feeabefb6b5.slice/crio-3df9ea77da67431ffd9f18a4be5f85afb5de18b6502ec78072438f579378a2e7 WatchSource:0}: Error finding container 3df9ea77da67431ffd9f18a4be5f85afb5de18b6502ec78072438f579378a2e7: Status 404 returned error can't find the container with id 3df9ea77da67431ffd9f18a4be5f85afb5de18b6502ec78072438f579378a2e7
Sep 09 12:33:49.422109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:49.422072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq" event={"ID":"928f8bee-de03-4fc3-afd7-2feeabefb6b5","Type":"ContainerStarted","Data":"3df9ea77da67431ffd9f18a4be5f85afb5de18b6502ec78072438f579378a2e7"}
Sep 09 12:33:49.438799 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:49.438767    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:50.104588 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:50.104550    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:50.170785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:50.170749    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:52.439453 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:52.439417    2579 generic.go:356] "Generic (PLEG): container finished" podID="928f8bee-de03-4fc3-afd7-2feeabefb6b5" containerID="de87abdf56d63aac5dbdccd50a4faf9623c64e678cbc9c8e0105dd0e9ef6465e" exitCode=0
Sep 09 12:33:52.439847 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:52.439477    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq" event={"ID":"928f8bee-de03-4fc3-afd7-2feeabefb6b5","Type":"ContainerDied","Data":"de87abdf56d63aac5dbdccd50a4faf9623c64e678cbc9c8e0105dd0e9ef6465e"}
Sep 09 12:33:52.988441 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:52.988408    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:33:53.566220 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:53.566191    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"
Sep 09 12:33:53.605281 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:53.605251    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928f8bee-de03-4fc3-afd7-2feeabefb6b5-maas-api-service-ca\") pod \"928f8bee-de03-4fc3-afd7-2feeabefb6b5\" (UID: \"928f8bee-de03-4fc3-afd7-2feeabefb6b5\") "
Sep 09 12:33:53.605616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:53.605591    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/928f8bee-de03-4fc3-afd7-2feeabefb6b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "928f8bee-de03-4fc3-afd7-2feeabefb6b5" (UID: "928f8bee-de03-4fc3-afd7-2feeabefb6b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:33:53.706176 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:53.706140    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928f8bee-de03-4fc3-afd7-2feeabefb6b5-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:33:54.449654 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:54.449609    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"
Sep 09 12:33:54.449813 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:54.449613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq" event={"ID":"928f8bee-de03-4fc3-afd7-2feeabefb6b5","Type":"ContainerDied","Data":"3df9ea77da67431ffd9f18a4be5f85afb5de18b6502ec78072438f579378a2e7"}
Sep 09 12:33:54.449813 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:54.449783    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3df9ea77da67431ffd9f18a4be5f85afb5de18b6502ec78072438f579378a2e7"
Sep 09 12:33:56.407032 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:56.406994    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:33:56.407601 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:33:56.407081    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:34:04.127419 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:04.127381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:04.751187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:04.751153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:04.823562 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:04.823524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:04.878945 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:04.878906    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:04.966333 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:04.966290    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"]
Sep 09 12:34:04.971645 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:04.971613    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f09b7dee-rzwvq"]
Sep 09 12:34:04.971949 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:04.971922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:05.054227 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:05.054134    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:05.107954 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:05.107908    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"]
Sep 09 12:34:05.108159 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:05.108129    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy" containerID="cri-o://4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317" gracePeriod=30
Sep 09 12:34:05.144302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:05.144274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:05.579367 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:05.579293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:05.672625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:05.672589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:05.774503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:05.774471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:05.875577 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:05.875468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:06.102533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:06.102497    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="928f8bee-de03-4fc3-afd7-2feeabefb6b5" path="/var/lib/kubelet/pods/928f8bee-de03-4fc3-afd7-2feeabefb6b5/volumes"
Sep 09 12:34:06.125283 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:06.125244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:06.198687 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:06.198619    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:07.017479 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:07.017438    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:08.894557 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:08.894524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:08.994752 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:08.994717    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:09.218321 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:09.218267    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:09.288526 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:09.288492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:09.393383 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:09.393346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:09.859575 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:09.859538    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:09.952257 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:09.952222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:10.380216 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.380143    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:34:10.451291 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451250    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-certs\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451301    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-token\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451325    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istiod-ca-cert\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451349    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-data\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451390    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-envoy\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451436    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-socket\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451482    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmtrx\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-kube-api-access-jmtrx\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451855 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451505    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-credential-socket\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451855 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451538    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-podinfo\") pod \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\" (UID: \"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e\") "
Sep 09 12:34:10.451855 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451589    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:10.451855 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451726    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:10.451855 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451800    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:34:10.452112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451919    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-certs\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.452112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451937    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istiod-ca-cert\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.452112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451940    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:10.452112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.451951    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-data\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.452112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.452022    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:10.453776 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.453753    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:34:10.453776 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.453761    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:34:10.453950 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.453929    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-kube-api-access-jmtrx" (OuterVolumeSpecName: "kube-api-access-jmtrx") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "kube-api-access-jmtrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:34:10.453989 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.453955    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" (UID: "ee75ee85-b41a-4b19-9e1c-14d02ed4c36e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:34:10.522018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.521979    2579 generic.go:356] "Generic (PLEG): container finished" podID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerID="4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317" exitCode=0
Sep 09 12:34:10.522213 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.522031    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"
Sep 09 12:34:10.522213 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.522073    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" event={"ID":"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e","Type":"ContainerDied","Data":"4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317"}
Sep 09 12:34:10.522213 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.522138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p" event={"ID":"ee75ee85-b41a-4b19-9e1c-14d02ed4c36e","Type":"ContainerDied","Data":"65fe1b7bbb36e12ef2eca455222c4d3136a764f9a5dca505504e3df63a1647ed"}
Sep 09 12:34:10.522213 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.522182    2579 scope.go:117] "RemoveContainer" containerID="4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317"
Sep 09 12:34:10.531668 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.531630    2579 scope.go:117] "RemoveContainer" containerID="7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e"
Sep 09 12:34:10.539858 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.539836    2579 scope.go:117] "RemoveContainer" containerID="4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317"
Sep 09 12:34:10.540178 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:34:10.540152    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317\": container with ID starting with 4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317 not found: ID does not exist" containerID="4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317"
Sep 09 12:34:10.540225 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.540188    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317"} err="failed to get container status \"4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317\": rpc error: code = NotFound desc = could not find container \"4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317\": container with ID starting with 4e4e17effba0c426dad71eb27d1eb0ea38bb5149151fd017f8729fe5788ae317 not found: ID does not exist"
Sep 09 12:34:10.540225 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.540221    2579 scope.go:117] "RemoveContainer" containerID="7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e"
Sep 09 12:34:10.540505 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:34:10.540483    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e\": container with ID starting with 7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e not found: ID does not exist" containerID="7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e"
Sep 09 12:34:10.540567 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.540512    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e"} err="failed to get container status \"7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e\": rpc error: code = NotFound desc = could not find container \"7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e\": container with ID starting with 7e5c31a1ff780ee02f501a2df4e01bfaa36d6aaa8241f5d25c19d0093cf0381e not found: ID does not exist"
Sep 09 12:34:10.543734 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.543702    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"]
Sep 09 12:34:10.547353 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.547329    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f09b7dee-openshift-default-86b54b7d44-wqw8p"]
Sep 09 12:34:10.552753 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.552731    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-token\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.552838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.552754    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-envoy\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.552838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.552766    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-workload-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.552838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.552774    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmtrx\" (UniqueName: \"kubernetes.io/projected/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-kube-api-access-jmtrx\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.552838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.552785    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-credential-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:10.552838 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:10.552793    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e-istio-podinfo\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:12.102232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:12.102196    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" path="/var/lib/kubelet/pods/ee75ee85-b41a-4b19-9e1c-14d02ed4c36e/volumes"
Sep 09 12:34:33.853147 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853110    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"]
Sep 09 12:34:33.853614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853485    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy"
Sep 09 12:34:33.853614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853496    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy"
Sep 09 12:34:33.853614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853503    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928f8bee-de03-4fc3-afd7-2feeabefb6b5" containerName="revoke-keys"
Sep 09 12:34:33.853614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853509    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="928f8bee-de03-4fc3-afd7-2feeabefb6b5" containerName="revoke-keys"
Sep 09 12:34:33.853614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853518    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy"
Sep 09 12:34:33.853614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853525    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy"
Sep 09 12:34:33.853614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853581    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy"
Sep 09 12:34:33.853614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.853592    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="928f8bee-de03-4fc3-afd7-2feeabefb6b5" containerName="revoke-keys"
Sep 09 12:34:33.860469 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.860453    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"
Sep 09 12:34:33.862567 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.862538    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:34:33.862818 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.862797    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:34:33.866218 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.866195    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"]
Sep 09 12:34:33.965142 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:33.965108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716ea56d-0830-40ff-b254-e02f91489138-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk\" (UID: \"716ea56d-0830-40ff-b254-e02f91489138\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"
Sep 09 12:34:34.066261 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:34.066223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716ea56d-0830-40ff-b254-e02f91489138-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk\" (UID: \"716ea56d-0830-40ff-b254-e02f91489138\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"
Sep 09 12:34:34.067043 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:34.067019    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716ea56d-0830-40ff-b254-e02f91489138-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk\" (UID: \"716ea56d-0830-40ff-b254-e02f91489138\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"
Sep 09 12:34:34.196037 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:34.195931    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"
Sep 09 12:34:34.468182 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:34.468155    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"]
Sep 09 12:34:34.476863 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:34:34.470509    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716ea56d_0830_40ff_b254_e02f91489138.slice/crio-f3d91ae9a78a99b2bac70b9145f4e199ce892e59d1247891ab68ae482a5a99d4 WatchSource:0}: Error finding container f3d91ae9a78a99b2bac70b9145f4e199ce892e59d1247891ab68ae482a5a99d4: Status 404 returned error can't find the container with id f3d91ae9a78a99b2bac70b9145f4e199ce892e59d1247891ab68ae482a5a99d4
Sep 09 12:34:34.632537 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:34.632497    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk" event={"ID":"716ea56d-0830-40ff-b254-e02f91489138","Type":"ContainerStarted","Data":"f3d91ae9a78a99b2bac70b9145f4e199ce892e59d1247891ab68ae482a5a99d4"}
Sep 09 12:34:35.638159 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:35.638120    2579 generic.go:356] "Generic (PLEG): container finished" podID="716ea56d-0830-40ff-b254-e02f91489138" containerID="90b9c32d31e17205a7534b6d4cf0641b49901bacd92870ff5a8256cd72cdc045" exitCode=0
Sep 09 12:34:35.638538 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:35.638181    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk" event={"ID":"716ea56d-0830-40ff-b254-e02f91489138","Type":"ContainerDied","Data":"90b9c32d31e17205a7534b6d4cf0641b49901bacd92870ff5a8256cd72cdc045"}
Sep 09 12:34:36.766228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:36.766204    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"
Sep 09 12:34:36.892749 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:36.892713    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716ea56d-0830-40ff-b254-e02f91489138-maas-api-service-ca\") pod \"716ea56d-0830-40ff-b254-e02f91489138\" (UID: \"716ea56d-0830-40ff-b254-e02f91489138\") "
Sep 09 12:34:36.893125 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:36.893077    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/716ea56d-0830-40ff-b254-e02f91489138-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "716ea56d-0830-40ff-b254-e02f91489138" (UID: "716ea56d-0830-40ff-b254-e02f91489138"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:34:36.993542 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:36.993488    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/716ea56d-0830-40ff-b254-e02f91489138-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:34:37.649317 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:37.649288    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"
Sep 09 12:34:37.649505 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:37.649290    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk" event={"ID":"716ea56d-0830-40ff-b254-e02f91489138","Type":"ContainerDied","Data":"f3d91ae9a78a99b2bac70b9145f4e199ce892e59d1247891ab68ae482a5a99d4"}
Sep 09 12:34:37.649505 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:37.649423    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3d91ae9a78a99b2bac70b9145f4e199ce892e59d1247891ab68ae482a5a99d4"
Sep 09 12:34:39.844519 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:39.844485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:39.914466 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:39.914435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:39.983171 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:39.983137    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:40.085984 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:40.085952    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:40.137525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:40.137441    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"]
Sep 09 12:34:40.141349 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:40.141322    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-dbdca9a8-dj2nk"]
Sep 09 12:34:40.144328 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:40.144300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:40.296984 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:40.296951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:40.746338 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:40.746300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:41.005157 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.005076    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"]
Sep 09 12:34:41.005589 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.005419    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="716ea56d-0830-40ff-b254-e02f91489138" containerName="revoke-keys"
Sep 09 12:34:41.005589 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.005430    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="716ea56d-0830-40ff-b254-e02f91489138" containerName="revoke-keys"
Sep 09 12:34:41.005589 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.005499    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee75ee85-b41a-4b19-9e1c-14d02ed4c36e" containerName="istio-proxy"
Sep 09 12:34:41.005589 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.005507    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="716ea56d-0830-40ff-b254-e02f91489138" containerName="revoke-keys"
Sep 09 12:34:41.008373 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.008356    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.010386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.010359    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-aea1bdeb-openshift-default-dockercfg-rxdh9\""
Sep 09 12:34:41.019290 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.019269    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"]
Sep 09 12:34:41.026177 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-socket\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.026312 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026183    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-credential-socket\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.026312 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-envoy\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.026312 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-podinfo\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.026491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-certs\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.026491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkk68\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-kube-api-access-pkk68\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.026491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026411    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26a037-649b-47fe-8f6f-920f76a66fca-istiod-ca-cert\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.026491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026440    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-data\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.026491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.026455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-token\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.065553 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.065520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:41.127438 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkk68\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-kube-api-access-pkk68\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.127727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26a037-649b-47fe-8f6f-920f76a66fca-istiod-ca-cert\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.127727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-data\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.127727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-token\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.127727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-socket\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.127727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-credential-socket\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.127727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127671    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-envoy\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.127727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-podinfo\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.127727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.127725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-certs\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.128394 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.128095    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-certs\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.128394 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.128294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26a037-649b-47fe-8f6f-920f76a66fca-istiod-ca-cert\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.128479 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.128454    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-credential-socket\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.128549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.128524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-socket\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.128814 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.128777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-data\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.130731 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.130712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-envoy\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.130869 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.130847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-podinfo\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.138969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.138946    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkk68\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-kube-api-access-pkk68\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.140152 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.140129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-token\") pod \"e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") " pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.166247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.166219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:41.319080 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.318992    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:41.595490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.595462    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"]
Sep 09 12:34:41.597932 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:34:41.597905    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d26a037_649b_47fe_8f6f_920f76a66fca.slice/crio-031382e82e2d0f8bb7ff5688638e56ba6d292b5ff9e73554d93d29d00ac4195e WatchSource:0}: Error finding container 031382e82e2d0f8bb7ff5688638e56ba6d292b5ff9e73554d93d29d00ac4195e: Status 404 returned error can't find the container with id 031382e82e2d0f8bb7ff5688638e56ba6d292b5ff9e73554d93d29d00ac4195e
Sep 09 12:34:41.600121 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.600089    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:34:41.600186 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.600153    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:34:41.600186 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.600180    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:34:41.682451 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.682413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" event={"ID":"6d26a037-649b-47fe-8f6f-920f76a66fca","Type":"ContainerStarted","Data":"23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d"}
Sep 09 12:34:41.682609 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.682504    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" event={"ID":"6d26a037-649b-47fe-8f6f-920f76a66fca","Type":"ContainerStarted","Data":"031382e82e2d0f8bb7ff5688638e56ba6d292b5ff9e73554d93d29d00ac4195e"}
Sep 09 12:34:41.702435 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:41.702351    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" podStartSLOduration=1.702329915 podStartE2EDuration="1.702329915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:34:41.700364525 +0000 UTC m=+898.024585056" watchObservedRunningTime="2026-09-09 12:34:41.702329915 +0000 UTC m=+898.026550447"
Sep 09 12:34:42.103043 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:42.103008    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="716ea56d-0830-40ff-b254-e02f91489138" path="/var/lib/kubelet/pods/716ea56d-0830-40ff-b254-e02f91489138/volumes"
Sep 09 12:34:42.319869 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:42.319834    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:42.324911 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:42.324887    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:42.686660 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:42.686611    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:42.687450 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:42.687431    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:34:43.048021 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:43.047991    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:43.149139 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:43.149103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:44.230777 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:44.230741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:44.330356 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:44.330321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:44.694140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:44.694105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:44.792195 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:44.792160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:34:44.859136 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:34:44.859102    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:01.795135 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:01.795101    2579 generic.go:356] "Generic (PLEG): container finished" podID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerID="23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d" exitCode=0
Sep 09 12:35:01.795585 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:01.795161    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" event={"ID":"6d26a037-649b-47fe-8f6f-920f76a66fca","Type":"ContainerDied","Data":"23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d"}
Sep 09 12:35:01.795585 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:01.795545    2579 scope.go:117] "RemoveContainer" containerID="23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d"
Sep 09 12:35:01.796567 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:01.796541    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:35:01.796625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:01.796609    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:35:01.796692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:01.796659    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:35:02.802350 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:02.802319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" event={"ID":"6d26a037-649b-47fe-8f6f-920f76a66fca","Type":"ContainerStarted","Data":"0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684"}
Sep 09 12:35:03.319344 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:03.319307    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:35:03.319549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:03.319531    2579 patch_prober.go:28] interesting pod/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 12:35:03.319600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:03.319577    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 12:35:04.320111 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:04.320076    2579 patch_prober.go:28] interesting pod/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 12:35:04.320520 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:04.320130    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 12:35:05.320374 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:05.320331    2579 patch_prober.go:28] interesting pod/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 12:35:05.320860 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:05.320392    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 12:35:06.320330 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:06.320273    2579 patch_prober.go:28] interesting pod/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 12:35:06.320534 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:06.320360    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 12:35:07.320222 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:07.320179    2579 patch_prober.go:28] interesting pod/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 09 12:35:07.320408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:07.320240    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 09 12:35:08.323342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.323303    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:35:08.323843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.323666    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:35:08.324280 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.324259    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:35:08.835946 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.835912    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"]
Sep 09 12:35:08.839387 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.839371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"
Sep 09 12:35:08.841522 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.841500    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:35:08.841656 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.841591    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:35:08.847107 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.847087    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"]
Sep 09 12:35:08.983865 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:08.983826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c8c4e4-f6b0-4c11-8d02-33fa7aed6761-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm\" (UID: \"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"
Sep 09 12:35:09.084767 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:09.084726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c8c4e4-f6b0-4c11-8d02-33fa7aed6761-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm\" (UID: \"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"
Sep 09 12:35:09.085350 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:09.085330    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c8c4e4-f6b0-4c11-8d02-33fa7aed6761-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm\" (UID: \"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"
Sep 09 12:35:09.150310 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:09.150229    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"
Sep 09 12:35:09.431892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:09.431863    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"]
Sep 09 12:35:09.434588 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:35:09.434555    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c8c4e4_f6b0_4c11_8d02_33fa7aed6761.slice/crio-593286a426bee79bbe3fe27944be132d8002d21729a0566febe171b1baa7f8c6 WatchSource:0}: Error finding container 593286a426bee79bbe3fe27944be132d8002d21729a0566febe171b1baa7f8c6: Status 404 returned error can't find the container with id 593286a426bee79bbe3fe27944be132d8002d21729a0566febe171b1baa7f8c6
Sep 09 12:35:09.834115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:09.834076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm" event={"ID":"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761","Type":"ContainerStarted","Data":"efc06c8335aa8d99af0771775c9201d7e712575a7a754abfc138fe1b3504f126"}
Sep 09 12:35:09.834311 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:09.834180    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm" event={"ID":"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761","Type":"ContainerStarted","Data":"593286a426bee79bbe3fe27944be132d8002d21729a0566febe171b1baa7f8c6"}
Sep 09 12:35:09.850511 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:09.850440    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm" podStartSLOduration=1.8504180890000002 podStartE2EDuration="1.850418089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:35:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:35:09.849075523 +0000 UTC m=+926.173296058" watchObservedRunningTime="2026-09-09 12:35:09.850418089 +0000 UTC m=+926.174638627"
Sep 09 12:35:10.840100 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:10.840062    2579 generic.go:356] "Generic (PLEG): container finished" podID="64c8c4e4-f6b0-4c11-8d02-33fa7aed6761" containerID="efc06c8335aa8d99af0771775c9201d7e712575a7a754abfc138fe1b3504f126" exitCode=0
Sep 09 12:35:10.840492 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:10.840129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm" event={"ID":"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761","Type":"ContainerDied","Data":"efc06c8335aa8d99af0771775c9201d7e712575a7a754abfc138fe1b3504f126"}
Sep 09 12:35:11.760760 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:11.760724    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:11.977141 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:11.977116    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"
Sep 09 12:35:12.110184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:12.110104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c8c4e4-f6b0-4c11-8d02-33fa7aed6761-maas-api-service-ca\") pod \"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761\" (UID: \"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761\") "
Sep 09 12:35:12.110484 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:12.110462    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c8c4e4-f6b0-4c11-8d02-33fa7aed6761-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64c8c4e4-f6b0-4c11-8d02-33fa7aed6761" (UID: "64c8c4e4-f6b0-4c11-8d02-33fa7aed6761"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:35:12.211338 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:12.211296    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c8c4e4-f6b0-4c11-8d02-33fa7aed6761-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:12.850222 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:12.850185    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm" event={"ID":"64c8c4e4-f6b0-4c11-8d02-33fa7aed6761","Type":"ContainerDied","Data":"593286a426bee79bbe3fe27944be132d8002d21729a0566febe171b1baa7f8c6"}
Sep 09 12:35:12.850222 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:12.850208    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"
Sep 09 12:35:12.850424 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:12.850244    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="593286a426bee79bbe3fe27944be132d8002d21729a0566febe171b1baa7f8c6"
Sep 09 12:35:14.980344 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:14.980304    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:15.058657 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:15.058602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:15.129472 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:15.129436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:15.232256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:15.232153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:15.240527 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:15.240497    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"]
Sep 09 12:35:15.242094 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:15.242075    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aea1bdeb-k7csm"]
Sep 09 12:35:15.308061 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:15.308026    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:16.108681 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:16.108629    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c8c4e4-f6b0-4c11-8d02-33fa7aed6761" path="/var/lib/kubelet/pods/64c8c4e4-f6b0-4c11-8d02-33fa7aed6761/volumes"
Sep 09 12:35:16.339424 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:16.339387    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"]
Sep 09 12:35:16.339676 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:16.339614    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy" containerID="cri-o://0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684" gracePeriod=30
Sep 09 12:35:16.348935 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:16.348906    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:16.793162 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:16.793122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:17.107882 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:17.107802    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:17.533040 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:17.533000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:19.192287 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:19.192251    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:19.294069 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:19.294035    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:20.380104 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:20.380056    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:20.465605 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:20.465574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:20.577036 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:20.577001    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:20.960090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:20.960052    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:21.051817 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.051787    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:21.585574 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.585552    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:35:21.692530 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692433    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-credential-socket\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692530 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692486    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkk68\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-kube-api-access-pkk68\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692530 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692516    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-token\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692570    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-envoy\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692599    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-data\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692697    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-socket\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692724    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-certs\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692789    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-podinfo\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692818    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26a037-649b-47fe-8f6f-920f76a66fca-istiod-ca-cert\") pod \"6d26a037-649b-47fe-8f6f-920f76a66fca\" (UID: \"6d26a037-649b-47fe-8f6f-920f76a66fca\") "
Sep 09 12:35:21.692834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692818    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:21.693193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.692972    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:21.693193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.693009    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:21.693193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.693046    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:21.693193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.693120    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-credential-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.693193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.693138    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-data\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.693193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.693152    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.693193 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.693165    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-workload-certs\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.693545 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.693301    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d26a037-649b-47fe-8f6f-920f76a66fca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:35:21.694926 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.694901    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:35:21.695038 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.694915    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:35:21.695038 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.694926    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-kube-api-access-pkk68" (OuterVolumeSpecName: "kube-api-access-pkk68") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "kube-api-access-pkk68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:21.695038 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.695003    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d26a037-649b-47fe-8f6f-920f76a66fca" (UID: "6d26a037-649b-47fe-8f6f-920f76a66fca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:35:21.794376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.794339    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-podinfo\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.794376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.794371    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26a037-649b-47fe-8f6f-920f76a66fca-istiod-ca-cert\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.794376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.794380    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkk68\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-kube-api-access-pkk68\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.794614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.794390    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-token\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.794614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.794399    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26a037-649b-47fe-8f6f-920f76a66fca-istio-envoy\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:21.902340 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.902304    2579 generic.go:356] "Generic (PLEG): container finished" podID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerID="0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684" exitCode=0
Sep 09 12:35:21.902495 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.902361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" event={"ID":"6d26a037-649b-47fe-8f6f-920f76a66fca","Type":"ContainerDied","Data":"0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684"}
Sep 09 12:35:21.902495 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.902364    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"
Sep 09 12:35:21.902495 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.902427    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm" event={"ID":"6d26a037-649b-47fe-8f6f-920f76a66fca","Type":"ContainerDied","Data":"031382e82e2d0f8bb7ff5688638e56ba6d292b5ff9e73554d93d29d00ac4195e"}
Sep 09 12:35:21.902495 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.902450    2579 scope.go:117] "RemoveContainer" containerID="0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684"
Sep 09 12:35:21.912691 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.912669    2579 scope.go:117] "RemoveContainer" containerID="23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d"
Sep 09 12:35:21.921161 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.921140    2579 scope.go:117] "RemoveContainer" containerID="0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684"
Sep 09 12:35:21.921466 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:35:21.921444    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684\": container with ID starting with 0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684 not found: ID does not exist" containerID="0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684"
Sep 09 12:35:21.921527 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.921477    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684"} err="failed to get container status \"0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684\": rpc error: code = NotFound desc = could not find container \"0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684\": container with ID starting with 0fd9a8bfc5c7189a8176bab22eb3f7a5698db705f7b12cea3400170fe3c34684 not found: ID does not exist"
Sep 09 12:35:21.921527 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.921500    2579 scope.go:117] "RemoveContainer" containerID="23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d"
Sep 09 12:35:21.921780 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:35:21.921762    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d\": container with ID starting with 23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d not found: ID does not exist" containerID="23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d"
Sep 09 12:35:21.921834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.921787    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d"} err="failed to get container status \"23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d\": rpc error: code = NotFound desc = could not find container \"23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d\": container with ID starting with 23b695f92e6224d6316f2ff514a35000d5bcb45848ede144d62b51605d47764d not found: ID does not exist"
Sep 09 12:35:21.927342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.927316    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"]
Sep 09 12:35:21.934628 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:21.934602    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-aea1bdeb-openshift-default-6c8bfccf78-29plm"]
Sep 09 12:35:22.102339 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:22.102296    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" path="/var/lib/kubelet/pods/6d26a037-649b-47fe-8f6f-920f76a66fca/volumes"
Sep 09 12:35:33.964895 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:33.964857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:41.403440 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:41.403403    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:44.090447 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090410    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"]
Sep 09 12:35:44.090874 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090763    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy"
Sep 09 12:35:44.090874 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090776    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy"
Sep 09 12:35:44.090874 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090785    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c8c4e4-f6b0-4c11-8d02-33fa7aed6761" containerName="revoke-keys"
Sep 09 12:35:44.090874 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090791    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c8c4e4-f6b0-4c11-8d02-33fa7aed6761" containerName="revoke-keys"
Sep 09 12:35:44.090874 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090808    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy"
Sep 09 12:35:44.090874 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090814    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy"
Sep 09 12:35:44.091058 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090886    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy"
Sep 09 12:35:44.091058 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090894    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d26a037-649b-47fe-8f6f-920f76a66fca" containerName="istio-proxy"
Sep 09 12:35:44.091058 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.090902    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c8c4e4-f6b0-4c11-8d02-33fa7aed6761" containerName="revoke-keys"
Sep 09 12:35:44.094314 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.094289    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"
Sep 09 12:35:44.096551 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.096527    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:35:44.096696 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.096594    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:35:44.105294 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.105270    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"]
Sep 09 12:35:44.182731 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.182691    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467548d7-84f9-4479-aeb7-691c850156fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ace3fa44-ql746\" (UID: \"467548d7-84f9-4479-aeb7-691c850156fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"
Sep 09 12:35:44.283514 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.283481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467548d7-84f9-4479-aeb7-691c850156fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ace3fa44-ql746\" (UID: \"467548d7-84f9-4479-aeb7-691c850156fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"
Sep 09 12:35:44.284232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.284207    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467548d7-84f9-4479-aeb7-691c850156fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ace3fa44-ql746\" (UID: \"467548d7-84f9-4479-aeb7-691c850156fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"
Sep 09 12:35:44.408762 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.408630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"
Sep 09 12:35:44.683306 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:44.683279    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"]
Sep 09 12:35:44.685932 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:35:44.685901    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467548d7_84f9_4479_aeb7_691c850156fd.slice/crio-66f63b32a2ddaf621609ddf66d1e69202f6e74e3ae14cf58026cffcb762d527c WatchSource:0}: Error finding container 66f63b32a2ddaf621609ddf66d1e69202f6e74e3ae14cf58026cffcb762d527c: Status 404 returned error can't find the container with id 66f63b32a2ddaf621609ddf66d1e69202f6e74e3ae14cf58026cffcb762d527c
Sep 09 12:35:45.021525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:45.021491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746" event={"ID":"467548d7-84f9-4479-aeb7-691c850156fd","Type":"ContainerStarted","Data":"0c208e0af48d97258458f541c732b095636335e15be58b38e8bc8fbdcbcd7ff5"}
Sep 09 12:35:45.021878 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:45.021851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746" event={"ID":"467548d7-84f9-4479-aeb7-691c850156fd","Type":"ContainerStarted","Data":"66f63b32a2ddaf621609ddf66d1e69202f6e74e3ae14cf58026cffcb762d527c"}
Sep 09 12:35:45.040094 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:45.040046    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746" podStartSLOduration=1.040031674 podStartE2EDuration="1.040031674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:35:45.039206143 +0000 UTC m=+961.363426675" watchObservedRunningTime="2026-09-09 12:35:45.040031674 +0000 UTC m=+961.364252204"
Sep 09 12:35:46.027767 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:46.027729    2579 generic.go:356] "Generic (PLEG): container finished" podID="467548d7-84f9-4479-aeb7-691c850156fd" containerID="0c208e0af48d97258458f541c732b095636335e15be58b38e8bc8fbdcbcd7ff5" exitCode=0
Sep 09 12:35:46.028148 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:46.027799    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746" event={"ID":"467548d7-84f9-4479-aeb7-691c850156fd","Type":"ContainerDied","Data":"0c208e0af48d97258458f541c732b095636335e15be58b38e8bc8fbdcbcd7ff5"}
Sep 09 12:35:47.162503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:47.162480    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"
Sep 09 12:35:47.210612 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:47.210578    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467548d7-84f9-4479-aeb7-691c850156fd-maas-api-service-ca\") pod \"467548d7-84f9-4479-aeb7-691c850156fd\" (UID: \"467548d7-84f9-4479-aeb7-691c850156fd\") "
Sep 09 12:35:47.210977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:47.210952    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467548d7-84f9-4479-aeb7-691c850156fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "467548d7-84f9-4479-aeb7-691c850156fd" (UID: "467548d7-84f9-4479-aeb7-691c850156fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:35:47.311843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:47.311756    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/467548d7-84f9-4479-aeb7-691c850156fd-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:35:48.037321 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:48.037292    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"
Sep 09 12:35:48.037321 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:48.037316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746" event={"ID":"467548d7-84f9-4479-aeb7-691c850156fd","Type":"ContainerDied","Data":"66f63b32a2ddaf621609ddf66d1e69202f6e74e3ae14cf58026cffcb762d527c"}
Sep 09 12:35:48.037569 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:48.037359    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66f63b32a2ddaf621609ddf66d1e69202f6e74e3ae14cf58026cffcb762d527c"
Sep 09 12:35:51.072200 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:51.072170    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:51.171662 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:51.171604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:51.262250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:51.262218    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"]
Sep 09 12:35:51.265214 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:51.265176    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ace3fa44-ql746"]
Sep 09 12:35:51.265472 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:51.265448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:51.323831 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:51.323730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:51.419891 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:51.419862    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:52.098112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:52.098076    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:52.103862 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:52.103837    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467548d7-84f9-4479-aeb7-691c850156fd" path="/var/lib/kubelet/pods/467548d7-84f9-4479-aeb7-691c850156fd/volumes"
Sep 09 12:35:52.210791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:52.210744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:52.308569 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:52.308537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:53.697965 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:53.697924    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:56.974630 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:56.974586    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:35:59.943290 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:35:59.943254    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:00.004758 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:00.004728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:00.083968 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:00.083932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:00.179139 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:00.179105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:00.453182 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:00.453148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:00.655515 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:00.655479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:05.113155 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:05.113109    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:13.562309 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:13.562224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:15.441382 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:15.441327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:16.241982 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:16.241949    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:17.056994 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.056960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:17.138682 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.138647    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"]
Sep 09 12:36:17.139056 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.139038    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467548d7-84f9-4479-aeb7-691c850156fd" containerName="revoke-keys"
Sep 09 12:36:17.139142 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.139058    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="467548d7-84f9-4479-aeb7-691c850156fd" containerName="revoke-keys"
Sep 09 12:36:17.139196 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.139164    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="467548d7-84f9-4479-aeb7-691c850156fd" containerName="revoke-keys"
Sep 09 12:36:17.142285 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.142263    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"
Sep 09 12:36:17.144468 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.144444    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:36:17.144571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.144447    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:36:17.149916 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.149895    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"]
Sep 09 12:36:17.266112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.266076    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:17.272965 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.272939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c59a5eca-9c02-4e74-bcb0-4a638d7bbdda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-e917b0-qsqkk\" (UID: \"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"
Sep 09 12:36:17.374600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.374504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c59a5eca-9c02-4e74-bcb0-4a638d7bbdda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-e917b0-qsqkk\" (UID: \"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"
Sep 09 12:36:17.375124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.375103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c59a5eca-9c02-4e74-bcb0-4a638d7bbdda-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-e917b0-qsqkk\" (UID: \"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"
Sep 09 12:36:17.453028 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.452992    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"
Sep 09 12:36:17.723469 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:17.723443    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"]
Sep 09 12:36:17.725432 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:36:17.725401    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59a5eca_9c02_4e74_bcb0_4a638d7bbdda.slice/crio-0cc96794e30cef8f6de34c08cba5b2744472d9c64ef3f5292288093283f229b8 WatchSource:0}: Error finding container 0cc96794e30cef8f6de34c08cba5b2744472d9c64ef3f5292288093283f229b8: Status 404 returned error can't find the container with id 0cc96794e30cef8f6de34c08cba5b2744472d9c64ef3f5292288093283f229b8
Sep 09 12:36:18.184496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:18.184460    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk" event={"ID":"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda","Type":"ContainerStarted","Data":"bbe13754d52405d712ed0ca00c529d02dbe3b4c2fab9587263a7d9958258f4d1"}
Sep 09 12:36:18.184875 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:18.184521    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk" event={"ID":"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda","Type":"ContainerStarted","Data":"0cc96794e30cef8f6de34c08cba5b2744472d9c64ef3f5292288093283f229b8"}
Sep 09 12:36:18.201199 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:18.201144    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk" podStartSLOduration=1.20112761 podStartE2EDuration="1.20112761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:36:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:36:18.199520878 +0000 UTC m=+994.523741409" watchObservedRunningTime="2026-09-09 12:36:18.20112761 +0000 UTC m=+994.525348141"
Sep 09 12:36:19.191697 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:19.191664    2579 generic.go:356] "Generic (PLEG): container finished" podID="c59a5eca-9c02-4e74-bcb0-4a638d7bbdda" containerID="bbe13754d52405d712ed0ca00c529d02dbe3b4c2fab9587263a7d9958258f4d1" exitCode=0
Sep 09 12:36:19.191697 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:19.191701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk" event={"ID":"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda","Type":"ContainerDied","Data":"bbe13754d52405d712ed0ca00c529d02dbe3b4c2fab9587263a7d9958258f4d1"}
Sep 09 12:36:20.344118 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:20.344093    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"
Sep 09 12:36:20.399764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:20.399735    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c59a5eca-9c02-4e74-bcb0-4a638d7bbdda-maas-api-service-ca\") pod \"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda\" (UID: \"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda\") "
Sep 09 12:36:20.400104 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:20.400084    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59a5eca-9c02-4e74-bcb0-4a638d7bbdda-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c59a5eca-9c02-4e74-bcb0-4a638d7bbdda" (UID: "c59a5eca-9c02-4e74-bcb0-4a638d7bbdda"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:36:20.500472 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:20.500438    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c59a5eca-9c02-4e74-bcb0-4a638d7bbdda-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:36:21.202377 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:21.202346    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"
Sep 09 12:36:21.202377 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:21.202369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk" event={"ID":"c59a5eca-9c02-4e74-bcb0-4a638d7bbdda","Type":"ContainerDied","Data":"0cc96794e30cef8f6de34c08cba5b2744472d9c64ef3f5292288093283f229b8"}
Sep 09 12:36:21.202582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:21.202417    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cc96794e30cef8f6de34c08cba5b2744472d9c64ef3f5292288093283f229b8"
Sep 09 12:36:26.657571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:26.657533    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:27.661054 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.661025    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:27.676444 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.676413    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"]
Sep 09 12:36:27.676688 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.676630    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc" podUID="41f64af0-57f8-4275-bd84-93815ffef074" containerName="payload-processing" containerID="cri-o://2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c" gracePeriod=30
Sep 09 12:36:27.684305 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.684279    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"]
Sep 09 12:36:27.684490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.684471    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp" podUID="45385990-76d4-41dc-b6fc-28b95c74702a" containerName="payload-pre-processing" containerID="cri-o://fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648" gracePeriod=30
Sep 09 12:36:27.761989 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.761960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:27.857329 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.857261    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"]
Sep 09 12:36:27.857903 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.857818    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:27.859834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.859810    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-e917b0-qsqkk"]
Sep 09 12:36:27.916594 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.916522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:27.937613 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.937593    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:36:27.955096 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.955061    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f64af0-57f8-4275-bd84-93815ffef074-plugins-config-volume\") pod \"41f64af0-57f8-4275-bd84-93815ffef074\" (UID: \"41f64af0-57f8-4275-bd84-93815ffef074\") "
Sep 09 12:36:27.955238 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.955109    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h97nk\" (UniqueName: \"kubernetes.io/projected/41f64af0-57f8-4275-bd84-93815ffef074-kube-api-access-h97nk\") pod \"41f64af0-57f8-4275-bd84-93815ffef074\" (UID: \"41f64af0-57f8-4275-bd84-93815ffef074\") "
Sep 09 12:36:27.955522 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.955486    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41f64af0-57f8-4275-bd84-93815ffef074-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41f64af0-57f8-4275-bd84-93815ffef074" (UID: "41f64af0-57f8-4275-bd84-93815ffef074"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:36:27.957419 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.957397    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41f64af0-57f8-4275-bd84-93815ffef074-kube-api-access-h97nk" (OuterVolumeSpecName: "kube-api-access-h97nk") pod "41f64af0-57f8-4275-bd84-93815ffef074" (UID: "41f64af0-57f8-4275-bd84-93815ffef074"). InnerVolumeSpecName "kube-api-access-h97nk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:36:27.959810 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:27.959790    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:36:28.056323 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.056291    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrlbv\" (UniqueName: \"kubernetes.io/projected/45385990-76d4-41dc-b6fc-28b95c74702a-kube-api-access-zrlbv\") pod \"45385990-76d4-41dc-b6fc-28b95c74702a\" (UID: \"45385990-76d4-41dc-b6fc-28b95c74702a\") "
Sep 09 12:36:28.056500 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.056347    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45385990-76d4-41dc-b6fc-28b95c74702a-plugins-config-volume\") pod \"45385990-76d4-41dc-b6fc-28b95c74702a\" (UID: \"45385990-76d4-41dc-b6fc-28b95c74702a\") "
Sep 09 12:36:28.056500 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.056468    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41f64af0-57f8-4275-bd84-93815ffef074-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:36:28.056500 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.056478    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h97nk\" (UniqueName: \"kubernetes.io/projected/41f64af0-57f8-4275-bd84-93815ffef074-kube-api-access-h97nk\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:36:28.056794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.056773    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45385990-76d4-41dc-b6fc-28b95c74702a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "45385990-76d4-41dc-b6fc-28b95c74702a" (UID: "45385990-76d4-41dc-b6fc-28b95c74702a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:36:28.058435 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.058418    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45385990-76d4-41dc-b6fc-28b95c74702a-kube-api-access-zrlbv" (OuterVolumeSpecName: "kube-api-access-zrlbv") pod "45385990-76d4-41dc-b6fc-28b95c74702a" (UID: "45385990-76d4-41dc-b6fc-28b95c74702a"). InnerVolumeSpecName "kube-api-access-zrlbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:36:28.102387 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.102352    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c59a5eca-9c02-4e74-bcb0-4a638d7bbdda" path="/var/lib/kubelet/pods/c59a5eca-9c02-4e74-bcb0-4a638d7bbdda/volumes"
Sep 09 12:36:28.157860 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.157818    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrlbv\" (UniqueName: \"kubernetes.io/projected/45385990-76d4-41dc-b6fc-28b95c74702a-kube-api-access-zrlbv\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:36:28.157860 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.157858    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/45385990-76d4-41dc-b6fc-28b95c74702a-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:36:28.185691 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.185571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:28.236085 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.236050    2579 generic.go:356] "Generic (PLEG): container finished" podID="45385990-76d4-41dc-b6fc-28b95c74702a" containerID="fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648" exitCode=0
Sep 09 12:36:28.236221 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.236122    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"
Sep 09 12:36:28.236221 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.236136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp" event={"ID":"45385990-76d4-41dc-b6fc-28b95c74702a","Type":"ContainerDied","Data":"fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648"}
Sep 09 12:36:28.236342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.236219    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp" event={"ID":"45385990-76d4-41dc-b6fc-28b95c74702a","Type":"ContainerDied","Data":"74eae468d689aa0ad80aa2c6eb0306653fd8f214cae82d4c16d2dbfbfb0813dc"}
Sep 09 12:36:28.236342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.236256    2579 scope.go:117] "RemoveContainer" containerID="fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648"
Sep 09 12:36:28.237727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.237699    2579 generic.go:356] "Generic (PLEG): container finished" podID="41f64af0-57f8-4275-bd84-93815ffef074" containerID="2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c" exitCode=0
Sep 09 12:36:28.237854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.237751    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"
Sep 09 12:36:28.237854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.237755    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc" event={"ID":"41f64af0-57f8-4275-bd84-93815ffef074","Type":"ContainerDied","Data":"2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c"}
Sep 09 12:36:28.237854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.237797    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc" event={"ID":"41f64af0-57f8-4275-bd84-93815ffef074","Type":"ContainerDied","Data":"2ede9af90971c7ede3c462f1d0c5b935daae5bad2d604ff21d0c380d32c9e572"}
Sep 09 12:36:28.245295 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.245278    2579 scope.go:117] "RemoveContainer" containerID="fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648"
Sep 09 12:36:28.245662 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:36:28.245620    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648\": container with ID starting with fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648 not found: ID does not exist" containerID="fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648"
Sep 09 12:36:28.245803 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.245660    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648"} err="failed to get container status \"fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648\": rpc error: code = NotFound desc = could not find container \"fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648\": container with ID starting with fbd92316379dfac6a7b264ba43bd90092be6859a27f8c5be897bc5b1b8245648 not found: ID does not exist"
Sep 09 12:36:28.245803 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.245679    2579 scope.go:117] "RemoveContainer" containerID="2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c"
Sep 09 12:36:28.253244 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.253227    2579 scope.go:117] "RemoveContainer" containerID="2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c"
Sep 09 12:36:28.253490 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:36:28.253468    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c\": container with ID starting with 2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c not found: ID does not exist" containerID="2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c"
Sep 09 12:36:28.253551 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.253500    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c"} err="failed to get container status \"2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c\": rpc error: code = NotFound desc = could not find container \"2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c\": container with ID starting with 2e2a7c9d470fcd2be414932e041df797773435ed8297fed4abac7e5f850f247c not found: ID does not exist"
Sep 09 12:36:28.256184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.256162    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"]
Sep 09 12:36:28.260828 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.260806    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-e917b0-5b6f44888f-qjxvc"]
Sep 09 12:36:28.270672 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.270620    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"]
Sep 09 12:36:28.276120 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:28.276101    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e917b0-65bd5967b6-wtsgp"]
Sep 09 12:36:30.104233 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:30.104193    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41f64af0-57f8-4275-bd84-93815ffef074" path="/var/lib/kubelet/pods/41f64af0-57f8-4275-bd84-93815ffef074/volumes"
Sep 09 12:36:30.104588 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:30.104545    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45385990-76d4-41dc-b6fc-28b95c74702a" path="/var/lib/kubelet/pods/45385990-76d4-41dc-b6fc-28b95c74702a/volumes"
Sep 09 12:36:33.068289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:33.068247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:33.169629 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:33.169597    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:33.268251 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:33.268215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:33.332033 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:33.331918    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:34.835736 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:34.835691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:34.935837 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:34.935805    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:35.116280 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.116152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:35.537428 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.537399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:35.794954 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.794823    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"]
Sep 09 12:36:35.795321 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795302    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41f64af0-57f8-4275-bd84-93815ffef074" containerName="payload-processing"
Sep 09 12:36:35.795386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795323    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="41f64af0-57f8-4275-bd84-93815ffef074" containerName="payload-processing"
Sep 09 12:36:35.795386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795332    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c59a5eca-9c02-4e74-bcb0-4a638d7bbdda" containerName="revoke-keys"
Sep 09 12:36:35.795386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795338    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c59a5eca-9c02-4e74-bcb0-4a638d7bbdda" containerName="revoke-keys"
Sep 09 12:36:35.795386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795349    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45385990-76d4-41dc-b6fc-28b95c74702a" containerName="payload-pre-processing"
Sep 09 12:36:35.795386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795356    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="45385990-76d4-41dc-b6fc-28b95c74702a" containerName="payload-pre-processing"
Sep 09 12:36:35.795559 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795407    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c59a5eca-9c02-4e74-bcb0-4a638d7bbdda" containerName="revoke-keys"
Sep 09 12:36:35.795559 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795416    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="41f64af0-57f8-4275-bd84-93815ffef074" containerName="payload-processing"
Sep 09 12:36:35.795559 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.795423    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="45385990-76d4-41dc-b6fc-28b95c74702a" containerName="payload-pre-processing"
Sep 09 12:36:35.799844 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.799822    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.802031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.802010    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-0f93fa-openshift-default-dockercfg-mkxtw\""
Sep 09 12:36:35.810592 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.810562    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"]
Sep 09 12:36:35.822674 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.822615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:35.827066 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-certs\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.827213 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827071    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfmr\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-kube-api-access-mjfmr\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.827213 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-data\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.827213 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-token\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.827348 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827263    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istiod-ca-cert\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.827348 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-socket\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.827491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-podinfo\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.827491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827431    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-envoy\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.827491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.827456    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-credential-socket\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928489 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istiod-ca-cert\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928499    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-socket\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-podinfo\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928653    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-envoy\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928694    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-credential-socket\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-certs\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfmr\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-kube-api-access-mjfmr\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-data\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.928969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.928825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-token\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.929417 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.929058    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-credential-socket\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.929417 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.929268    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-certs\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.929417 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.929325    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-data\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.929417 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.929355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istiod-ca-cert\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.929601 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.929446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-socket\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.931054 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.931036    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-podinfo\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.931377 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.931357    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-envoy\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.937371 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.937344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-token\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:35.937622 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:35.937598    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfmr\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-kube-api-access-mjfmr\") pod \"e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") " pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:36.110910 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:36.110829    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:36.385503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:36.385477    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"]
Sep 09 12:36:36.387423 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:36:36.387397    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f4f8c06_0d24_4e28_bf89_746be764dd6a.slice/crio-7dbb64044f91a306f691ea0e56f029930c770827496178c5900abd8502f684d4 WatchSource:0}: Error finding container 7dbb64044f91a306f691ea0e56f029930c770827496178c5900abd8502f684d4: Status 404 returned error can't find the container with id 7dbb64044f91a306f691ea0e56f029930c770827496178c5900abd8502f684d4
Sep 09 12:36:36.389592 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:36.389559    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:36:36.389681 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:36.389660    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:36:36.389733 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:36.389717    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:36:36.407009 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:36.406959    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:37.285581 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:37.285543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" event={"ID":"0f4f8c06-0d24-4e28-bf89-746be764dd6a","Type":"ContainerStarted","Data":"a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27"}
Sep 09 12:36:37.286094 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:37.285602    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" event={"ID":"0f4f8c06-0d24-4e28-bf89-746be764dd6a","Type":"ContainerStarted","Data":"7dbb64044f91a306f691ea0e56f029930c770827496178c5900abd8502f684d4"}
Sep 09 12:36:37.305534 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:37.305467    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" podStartSLOduration=2.305450234 podStartE2EDuration="2.305450234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:36:37.304279988 +0000 UTC m=+1013.628500517" watchObservedRunningTime="2026-09-09 12:36:37.305450234 +0000 UTC m=+1013.629670764"
Sep 09 12:36:38.111171 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:38.111138    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:38.116587 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:38.116562    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:38.290831 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:38.290791    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:38.291867 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:38.291850    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:39.194479 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:39.194443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:39.294289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:39.294255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:40.397302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:40.397263    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:40.455850 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:40.455811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:41.003527 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:41.003495    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:41.090624 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:41.090578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:41.268083 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:41.268004    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:49.396500 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:49.396455    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:51.119824 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:51.119728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:51.868206 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:51.868151    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:53.828208 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:53.828175    2579 patch_prober.go:28] interesting pod/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:38858->10.134.0.38:15021: read: connection reset by peer" start-of-body=
Sep 09 12:36:53.828830 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:53.828248    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": read tcp 10.134.0.2:38858->10.134.0.38:15021: read: connection reset by peer"
Sep 09 12:36:54.371164 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:54.371129    2579 generic.go:356] "Generic (PLEG): container finished" podID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerID="a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27" exitCode=0
Sep 09 12:36:54.371164 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:54.371165    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" event={"ID":"0f4f8c06-0d24-4e28-bf89-746be764dd6a","Type":"ContainerDied","Data":"a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27"}
Sep 09 12:36:54.371496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:54.371484    2579 scope.go:117] "RemoveContainer" containerID="a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27"
Sep 09 12:36:54.372492 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:54.372466    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:36:54.372547 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:54.372533    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:36:54.372593 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:54.372569    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:36:55.377652 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:55.377604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" event={"ID":"0f4f8c06-0d24-4e28-bf89-746be764dd6a","Type":"ContainerStarted","Data":"b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4"}
Sep 09 12:36:55.592203 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:55.592173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:55.658875 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:55.658778    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:55.741056 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:55.741016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:55.846053 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:55.846019    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:36:56.111319 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:56.111290    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:36:56.111545 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:56.111523    2579 patch_prober.go:28] interesting pod/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:36:56.111592 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:56.111570    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 09 12:36:57.112168 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:57.112130    2579 patch_prober.go:28] interesting pod/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:36:57.112559 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:57.112188    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 09 12:36:58.112007 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:58.111972    2579 patch_prober.go:28] interesting pod/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:36:58.112254 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:58.112024    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 09 12:36:59.111512 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:59.111467    2579 patch_prober.go:28] interesting pod/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:36:59.111703 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:36:59.111532    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 09 12:37:00.111289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:00.111253    2579 patch_prober.go:28] interesting pod/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 09 12:37:00.111712 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:00.111318    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 09 12:37:01.115333 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:01.115304    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:37:01.115794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:01.115684    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:37:01.116369 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:01.116355    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:37:04.176629 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:04.176590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:06.311520 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:06.311491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:07.091237 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:07.091200    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:13.569325 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:13.569290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:13.635538 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:13.635500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:13.696425 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:13.696388    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:13.798017 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:13.797981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:13.903491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:13.903403    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:14.501191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:14.501153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:14.590746 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:14.590709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:14.684615 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:14.684574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:16.166891 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:16.166856    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:21.654274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:21.654244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:22.727290 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:22.727249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:22.826114 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:22.826075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:23.460287 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:23.460255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:29.605609 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:29.605576    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:29.679350 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:29.679313    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:29.816423 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:29.816388    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:30.129158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:30.129127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:31.034159 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:31.034121    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:33.702322 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:33.702285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:33.801302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:33.801259    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:34.876391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:34.876354    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:34.951126 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:34.951092    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:35.051927 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.051895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:35.391438 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.391383    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:35.486779 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.486746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:35.498149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.498120    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"]
Sep 09 12:37:35.500319 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.500303    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:35.502221 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.502202    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2475a5c7\""
Sep 09 12:37:35.512247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.512223    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"]
Sep 09 12:37:35.541875 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.541842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh\" (UID: \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:35.542033 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.541982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2g4n\" (UniqueName: \"kubernetes.io/projected/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-kube-api-access-z2g4n\") pod \"payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh\" (UID: \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:35.642606 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.642497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2g4n\" (UniqueName: \"kubernetes.io/projected/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-kube-api-access-z2g4n\") pod \"payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh\" (UID: \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:35.642606 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.642573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh\" (UID: \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:35.643537 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.643459    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh\" (UID: \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:35.650712 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.650678    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"]
Sep 09 12:37:35.654039 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.654000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2g4n\" (UniqueName: \"kubernetes.io/projected/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-kube-api-access-z2g4n\") pod \"payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh\" (UID: \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:35.662914 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.662886    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"]
Sep 09 12:37:35.663050 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.663000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:35.665289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.665268    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2475a5c7-dockercfg-jsl4d\""
Sep 09 12:37:35.744033 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.743998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzkgg\" (UniqueName: \"kubernetes.io/projected/31c87568-5b94-4d52-880c-f1af968ec7cb-kube-api-access-jzkgg\") pod \"payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns\" (UID: \"31c87568-5b94-4d52-880c-f1af968ec7cb\") " pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:35.744225 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.744045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31c87568-5b94-4d52-880c-f1af968ec7cb-plugins-config-volume\") pod \"payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns\" (UID: \"31c87568-5b94-4d52-880c-f1af968ec7cb\") " pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:35.810632 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.810594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:35.844774 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.844731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkgg\" (UniqueName: \"kubernetes.io/projected/31c87568-5b94-4d52-880c-f1af968ec7cb-kube-api-access-jzkgg\") pod \"payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns\" (UID: \"31c87568-5b94-4d52-880c-f1af968ec7cb\") " pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:35.844929 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.844788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31c87568-5b94-4d52-880c-f1af968ec7cb-plugins-config-volume\") pod \"payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns\" (UID: \"31c87568-5b94-4d52-880c-f1af968ec7cb\") " pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:35.845915 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.845878    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31c87568-5b94-4d52-880c-f1af968ec7cb-plugins-config-volume\") pod \"payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns\" (UID: \"31c87568-5b94-4d52-880c-f1af968ec7cb\") " pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:35.853684 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.853658    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzkgg\" (UniqueName: \"kubernetes.io/projected/31c87568-5b94-4d52-880c-f1af968ec7cb-kube-api-access-jzkgg\") pod \"payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns\" (UID: \"31c87568-5b94-4d52-880c-f1af968ec7cb\") " pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:35.973541 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:35.973512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:36.078051 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.077981    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"]
Sep 09 12:37:36.080279 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:37:36.080250    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b688bbe_fe7f_4dbe_a158_e472f4247ed9.slice/crio-5d4a0dffde07a7cab20cff949d6ad9ad8a2b236faf7c5f027b48d069fcca7d7d WatchSource:0}: Error finding container 5d4a0dffde07a7cab20cff949d6ad9ad8a2b236faf7c5f027b48d069fcca7d7d: Status 404 returned error can't find the container with id 5d4a0dffde07a7cab20cff949d6ad9ad8a2b236faf7c5f027b48d069fcca7d7d
Sep 09 12:37:36.243769 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.243744    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"]
Sep 09 12:37:36.246042 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:37:36.246010    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c87568_5b94_4d52_880c_f1af968ec7cb.slice/crio-b194495d78ab4af71b0d0e203c74ab6179db5644051abf58ec5aee57cf7dcce0 WatchSource:0}: Error finding container b194495d78ab4af71b0d0e203c74ab6179db5644051abf58ec5aee57cf7dcce0: Status 404 returned error can't find the container with id b194495d78ab4af71b0d0e203c74ab6179db5644051abf58ec5aee57cf7dcce0
Sep 09 12:37:36.578106 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.578001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh" event={"ID":"0b688bbe-fe7f-4dbe-a158-e472f4247ed9","Type":"ContainerStarted","Data":"34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01"}
Sep 09 12:37:36.578106 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.578080    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:36.578106 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.578105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh" event={"ID":"0b688bbe-fe7f-4dbe-a158-e472f4247ed9","Type":"ContainerStarted","Data":"5d4a0dffde07a7cab20cff949d6ad9ad8a2b236faf7c5f027b48d069fcca7d7d"}
Sep 09 12:37:36.579784 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.579755    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns" event={"ID":"31c87568-5b94-4d52-880c-f1af968ec7cb","Type":"ContainerStarted","Data":"9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead"}
Sep 09 12:37:36.579898 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.579806    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:36.579898 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.579830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns" event={"ID":"31c87568-5b94-4d52-880c-f1af968ec7cb","Type":"ContainerStarted","Data":"b194495d78ab4af71b0d0e203c74ab6179db5644051abf58ec5aee57cf7dcce0"}
Sep 09 12:37:36.595109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.595062    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh" podStartSLOduration=1.5950482940000001 podStartE2EDuration="1.595048294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:37:36.593123302 +0000 UTC m=+1072.917343846" watchObservedRunningTime="2026-09-09 12:37:36.595048294 +0000 UTC m=+1072.919268859"
Sep 09 12:37:36.612365 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:36.612308    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns" podStartSLOduration=1.612279979 podStartE2EDuration="1.612279979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:37:36.610036361 +0000 UTC m=+1072.934256884" watchObservedRunningTime="2026-09-09 12:37:36.612279979 +0000 UTC m=+1072.936500512"
Sep 09 12:37:47.586819 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:47.586779    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:37:47.587218 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:47.586861    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:37:50.488058 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.488018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:50.726690 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.726654    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"]
Sep 09 12:37:50.728817 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.728801    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"
Sep 09 12:37:50.731181 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.731157    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:37:50.731298 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.731160    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:37:50.737217 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.737192    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"]
Sep 09 12:37:50.866315 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.866213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:50.869127 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.869104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg\" (UID: \"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"
Sep 09 12:37:50.970098 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.970060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg\" (UID: \"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"
Sep 09 12:37:50.970711 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:50.970692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg\" (UID: \"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"
Sep 09 12:37:51.039742 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:51.039704    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"
Sep 09 12:37:51.308674 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:51.308626    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"]
Sep 09 12:37:51.310960 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:37:51.310935    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod304ca0d1_c4f0_47f7_9048_6bf1cdf9ef7a.slice/crio-968f4db637f4081d199b013d3f6886e0b62beb1da445bb5a81005ca87aa6674b WatchSource:0}: Error finding container 968f4db637f4081d199b013d3f6886e0b62beb1da445bb5a81005ca87aa6674b: Status 404 returned error can't find the container with id 968f4db637f4081d199b013d3f6886e0b62beb1da445bb5a81005ca87aa6674b
Sep 09 12:37:51.654187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:51.654093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg" event={"ID":"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a","Type":"ContainerStarted","Data":"7eaef7eb24aaf45435cb739f27e537f2d209ec458eb98cf38fd3888d7801f4e3"}
Sep 09 12:37:51.654578 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:51.654204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg" event={"ID":"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a","Type":"ContainerStarted","Data":"968f4db637f4081d199b013d3f6886e0b62beb1da445bb5a81005ca87aa6674b"}
Sep 09 12:37:51.669681 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:51.669612    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg" podStartSLOduration=1.669590879 podStartE2EDuration="1.669590879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:37:51.667835439 +0000 UTC m=+1087.992055971" watchObservedRunningTime="2026-09-09 12:37:51.669590879 +0000 UTC m=+1087.993811412"
Sep 09 12:37:52.660473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:52.660431    2579 generic.go:356] "Generic (PLEG): container finished" podID="304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a" containerID="7eaef7eb24aaf45435cb739f27e537f2d209ec458eb98cf38fd3888d7801f4e3" exitCode=0
Sep 09 12:37:52.660932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:52.660507    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg" event={"ID":"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a","Type":"ContainerDied","Data":"7eaef7eb24aaf45435cb739f27e537f2d209ec458eb98cf38fd3888d7801f4e3"}
Sep 09 12:37:53.539014 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:53.538985    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:53.790980 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:53.790921    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"
Sep 09 12:37:53.897303 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:53.897264    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a-maas-api-service-ca\") pod \"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a\" (UID: \"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a\") "
Sep 09 12:37:53.897626 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:53.897601    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a" (UID: "304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:37:53.998336 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:53.998299    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:37:54.364272 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:54.364185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:54.671934 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:54.671850    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"
Sep 09 12:37:54.672097 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:54.671849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg" event={"ID":"304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a","Type":"ContainerDied","Data":"968f4db637f4081d199b013d3f6886e0b62beb1da445bb5a81005ca87aa6674b"}
Sep 09 12:37:54.672097 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:54.671994    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="968f4db637f4081d199b013d3f6886e0b62beb1da445bb5a81005ca87aa6674b"
Sep 09 12:37:55.757168 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.757135    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:55.861852 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.861819    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"]
Sep 09 12:37:55.862162 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.862151    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a" containerName="revoke-keys"
Sep 09 12:37:55.862220 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.862164    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a" containerName="revoke-keys"
Sep 09 12:37:55.862257 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.862242    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a" containerName="revoke-keys"
Sep 09 12:37:55.864440 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.864417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"
Sep 09 12:37:55.866519 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.866501    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:37:55.866721 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.866691    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:37:55.873884 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.873851    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"]
Sep 09 12:37:55.893899 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:55.893854    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:56.016063 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:56.015952    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed38d896-6632-4c00-ae21-38ef58a836c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js\" (UID: \"ed38d896-6632-4c00-ae21-38ef58a836c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"
Sep 09 12:37:56.117349 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:56.117307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed38d896-6632-4c00-ae21-38ef58a836c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js\" (UID: \"ed38d896-6632-4c00-ae21-38ef58a836c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"
Sep 09 12:37:56.117991 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:56.117967    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed38d896-6632-4c00-ae21-38ef58a836c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js\" (UID: \"ed38d896-6632-4c00-ae21-38ef58a836c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"
Sep 09 12:37:56.176895 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:56.176862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"
Sep 09 12:37:56.451523 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:56.451496    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"]
Sep 09 12:37:56.453962 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:37:56.453937    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded38d896_6632_4c00_ae21_38ef58a836c8.slice/crio-09d13e4623160a10da0c17598f2f9dda410261d7db23c34c94efaba3dea087d8 WatchSource:0}: Error finding container 09d13e4623160a10da0c17598f2f9dda410261d7db23c34c94efaba3dea087d8: Status 404 returned error can't find the container with id 09d13e4623160a10da0c17598f2f9dda410261d7db23c34c94efaba3dea087d8
Sep 09 12:37:56.682597 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:56.682560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js" event={"ID":"ed38d896-6632-4c00-ae21-38ef58a836c8","Type":"ContainerStarted","Data":"09d13e4623160a10da0c17598f2f9dda410261d7db23c34c94efaba3dea087d8"}
Sep 09 12:37:56.720351 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:56.720264    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:57.688976 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:57.688944    2579 generic.go:356] "Generic (PLEG): container finished" podID="ed38d896-6632-4c00-ae21-38ef58a836c8" containerID="cc8bebff0060f0f917fdc910e1c6d2da67da7529a005900b24da034db8b3cf67" exitCode=0
Sep 09 12:37:57.689371 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:57.689028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js" event={"ID":"ed38d896-6632-4c00-ae21-38ef58a836c8","Type":"ContainerDied","Data":"cc8bebff0060f0f917fdc910e1c6d2da67da7529a005900b24da034db8b3cf67"}
Sep 09 12:37:57.705781 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:57.705739    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:58.755092 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:58.755063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:58.829760 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:58.829736    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"
Sep 09 12:37:58.944115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:58.944082    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed38d896-6632-4c00-ae21-38ef58a836c8-maas-api-service-ca\") pod \"ed38d896-6632-4c00-ae21-38ef58a836c8\" (UID: \"ed38d896-6632-4c00-ae21-38ef58a836c8\") "
Sep 09 12:37:58.944440 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:58.944391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed38d896-6632-4c00-ae21-38ef58a836c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed38d896-6632-4c00-ae21-38ef58a836c8" (UID: "ed38d896-6632-4c00-ae21-38ef58a836c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:37:59.045549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:59.045511    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed38d896-6632-4c00-ae21-38ef58a836c8-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:37:59.462003 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:59.461959    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:59.562265 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:59.562226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:37:59.700172 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:59.700143    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"
Sep 09 12:37:59.700172 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:59.700165    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js" event={"ID":"ed38d896-6632-4c00-ae21-38ef58a836c8","Type":"ContainerDied","Data":"09d13e4623160a10da0c17598f2f9dda410261d7db23c34c94efaba3dea087d8"}
Sep 09 12:37:59.700387 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:59.700208    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09d13e4623160a10da0c17598f2f9dda410261d7db23c34c94efaba3dea087d8"
Sep 09 12:37:59.850142 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:37:59.850057    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:00.871139 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:00.871093    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:01.593761 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:01.593709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:01.690402 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:01.690366    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"]
Sep 09 12:38:01.693540 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:01.693504    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0f93fa-p7cpg"]
Sep 09 12:38:01.693737 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:01.693629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:01.820545 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:01.820502    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:02.065270 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:02.065233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:02.103132 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:02.103097    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a" path="/var/lib/kubelet/pods/304ca0d1-c4f0-47f7-9048-6bf1cdf9ef7a/volumes"
Sep 09 12:38:06.189285 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.189250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:06.289282 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.289247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:06.390670 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.390618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:06.545565 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.545530    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"]
Sep 09 12:38:06.545917 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.545904    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed38d896-6632-4c00-ae21-38ef58a836c8" containerName="revoke-keys"
Sep 09 12:38:06.545964 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.545919    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed38d896-6632-4c00-ae21-38ef58a836c8" containerName="revoke-keys"
Sep 09 12:38:06.546000 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.545984    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed38d896-6632-4c00-ae21-38ef58a836c8" containerName="revoke-keys"
Sep 09 12:38:06.549152 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.549136    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"
Sep 09 12:38:06.551360 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.551339    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:38:06.551452 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.551344    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:38:06.557032 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.557004    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"]
Sep 09 12:38:06.717448 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.717403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40acd1c2-94fb-4395-a126-75b9653ff07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t\" (UID: \"40acd1c2-94fb-4395-a126-75b9653ff07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"
Sep 09 12:38:06.818839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.818752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40acd1c2-94fb-4395-a126-75b9653ff07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t\" (UID: \"40acd1c2-94fb-4395-a126-75b9653ff07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"
Sep 09 12:38:06.819342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.819322    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40acd1c2-94fb-4395-a126-75b9653ff07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t\" (UID: \"40acd1c2-94fb-4395-a126-75b9653ff07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"
Sep 09 12:38:06.860566 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:06.860526    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"
Sep 09 12:38:07.134025 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.133999    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"]
Sep 09 12:38:07.136092 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.136054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:07.136092 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:38:07.136076    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40acd1c2_94fb_4395_a126_75b9653ff07c.slice/crio-12ccf1ab68bfa3baec6aa368b9d55055a5bc7d58247590f0f645a4a0d7c93e34 WatchSource:0}: Error finding container 12ccf1ab68bfa3baec6aa368b9d55055a5bc7d58247590f0f645a4a0d7c93e34: Status 404 returned error can't find the container with id 12ccf1ab68bfa3baec6aa368b9d55055a5bc7d58247590f0f645a4a0d7c93e34
Sep 09 12:38:07.699868 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.699836    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:07.735482 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.735398    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"]
Sep 09 12:38:07.735671 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.735629    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c" podUID="641f1344-1baa-4b01-b74b-9f9f45c26d9c" containerName="payload-processing" containerID="cri-o://e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e" gracePeriod=30
Sep 09 12:38:07.745000 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.744966    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"]
Sep 09 12:38:07.745204 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.745179    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl" podUID="712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6" containerName="payload-pre-processing" containerID="cri-o://fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2" gracePeriod=30
Sep 09 12:38:07.745358 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.745331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t" event={"ID":"40acd1c2-94fb-4395-a126-75b9653ff07c","Type":"ContainerStarted","Data":"3f686ac5d7accc49da5e2f1c88a21557a11473e5664bb5539ef37ad3ff5a72f2"}
Sep 09 12:38:07.745448 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.745383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t" event={"ID":"40acd1c2-94fb-4395-a126-75b9653ff07c","Type":"ContainerStarted","Data":"12ccf1ab68bfa3baec6aa368b9d55055a5bc7d58247590f0f645a4a0d7c93e34"}
Sep 09 12:38:07.778738 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.778704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:07.787853 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.787795    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t" podStartSLOduration=1.7877748 podStartE2EDuration="1.7877748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:07.787150576 +0000 UTC m=+1104.111371100" watchObservedRunningTime="2026-09-09 12:38:07.7877748 +0000 UTC m=+1104.111995332"
Sep 09 12:38:07.835589 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.835561    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:07.916746 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.916716    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"]
Sep 09 12:38:07.919462 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.919437    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-1e223f-8x7js"]
Sep 09 12:38:07.920542 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.920515    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:07.985744 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:07.985654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:08.030097 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.030073    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:38:08.032504 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.032487    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:38:08.102704 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.102664    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed38d896-6632-4c00-ae21-38ef58a836c8" path="/var/lib/kubelet/pods/ed38d896-6632-4c00-ae21-38ef58a836c8/volumes"
Sep 09 12:38:08.132152 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.132124    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxdwc\" (UniqueName: \"kubernetes.io/projected/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-kube-api-access-dxdwc\") pod \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\" (UID: \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\") "
Sep 09 12:38:08.132347 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.132187    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzfnp\" (UniqueName: \"kubernetes.io/projected/641f1344-1baa-4b01-b74b-9f9f45c26d9c-kube-api-access-vzfnp\") pod \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\" (UID: \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\") "
Sep 09 12:38:08.132347 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.132227    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/641f1344-1baa-4b01-b74b-9f9f45c26d9c-plugins-config-volume\") pod \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\" (UID: \"641f1344-1baa-4b01-b74b-9f9f45c26d9c\") "
Sep 09 12:38:08.132474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.132364    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-plugins-config-volume\") pod \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\" (UID: \"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6\") "
Sep 09 12:38:08.132534 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.132494    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/641f1344-1baa-4b01-b74b-9f9f45c26d9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "641f1344-1baa-4b01-b74b-9f9f45c26d9c" (UID: "641f1344-1baa-4b01-b74b-9f9f45c26d9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:08.132750 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.132719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6" (UID: "712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:08.132880 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.132768    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/641f1344-1baa-4b01-b74b-9f9f45c26d9c-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:08.134319 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.134298    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-kube-api-access-dxdwc" (OuterVolumeSpecName: "kube-api-access-dxdwc") pod "712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6" (UID: "712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6"). InnerVolumeSpecName "kube-api-access-dxdwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:08.134319 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.134308    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641f1344-1baa-4b01-b74b-9f9f45c26d9c-kube-api-access-vzfnp" (OuterVolumeSpecName: "kube-api-access-vzfnp") pod "641f1344-1baa-4b01-b74b-9f9f45c26d9c" (UID: "641f1344-1baa-4b01-b74b-9f9f45c26d9c"). InnerVolumeSpecName "kube-api-access-vzfnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:08.233621 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.233584    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxdwc\" (UniqueName: \"kubernetes.io/projected/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-kube-api-access-dxdwc\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:08.233621 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.233616    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzfnp\" (UniqueName: \"kubernetes.io/projected/641f1344-1baa-4b01-b74b-9f9f45c26d9c-kube-api-access-vzfnp\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:08.233621 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.233625    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:08.268119 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.268029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:08.751860 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.751820    2579 generic.go:356] "Generic (PLEG): container finished" podID="641f1344-1baa-4b01-b74b-9f9f45c26d9c" containerID="e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e" exitCode=0
Sep 09 12:38:08.752340 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.751866    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"
Sep 09 12:38:08.752340 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.751901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c" event={"ID":"641f1344-1baa-4b01-b74b-9f9f45c26d9c","Type":"ContainerDied","Data":"e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e"}
Sep 09 12:38:08.752340 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.751971    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c" event={"ID":"641f1344-1baa-4b01-b74b-9f9f45c26d9c","Type":"ContainerDied","Data":"6debf013544c83cc30e38624bfcad7c7088956acc4b1a07895710f433ac29652"}
Sep 09 12:38:08.752340 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.752007    2579 scope.go:117] "RemoveContainer" containerID="e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e"
Sep 09 12:38:08.753559 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.753535    2579 generic.go:356] "Generic (PLEG): container finished" podID="40acd1c2-94fb-4395-a126-75b9653ff07c" containerID="3f686ac5d7accc49da5e2f1c88a21557a11473e5664bb5539ef37ad3ff5a72f2" exitCode=0
Sep 09 12:38:08.753697 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.753562    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t" event={"ID":"40acd1c2-94fb-4395-a126-75b9653ff07c","Type":"ContainerDied","Data":"3f686ac5d7accc49da5e2f1c88a21557a11473e5664bb5539ef37ad3ff5a72f2"}
Sep 09 12:38:08.755042 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.755020    2579 generic.go:356] "Generic (PLEG): container finished" podID="712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6" containerID="fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2" exitCode=0
Sep 09 12:38:08.755140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.755047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl" event={"ID":"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6","Type":"ContainerDied","Data":"fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2"}
Sep 09 12:38:08.755140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.755071    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"
Sep 09 12:38:08.755140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.755094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl" event={"ID":"712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6","Type":"ContainerDied","Data":"739dca7ea6d0fa24b24bdcc6615635631d2f1f3884c12e99e7578c5c9a0097db"}
Sep 09 12:38:08.763028 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.763008    2579 scope.go:117] "RemoveContainer" containerID="e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e"
Sep 09 12:38:08.763281 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:38:08.763265    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e\": container with ID starting with e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e not found: ID does not exist" containerID="e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e"
Sep 09 12:38:08.763354 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.763288    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e"} err="failed to get container status \"e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e\": rpc error: code = NotFound desc = could not find container \"e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e\": container with ID starting with e3d763a7cb735a4eed8dd2dd69cae67626e3eaeaf7a00a13449531e57e76ac8e not found: ID does not exist"
Sep 09 12:38:08.763354 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.763304    2579 scope.go:117] "RemoveContainer" containerID="fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2"
Sep 09 12:38:08.771120 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.771105    2579 scope.go:117] "RemoveContainer" containerID="fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2"
Sep 09 12:38:08.771374 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:38:08.771356    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2\": container with ID starting with fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2 not found: ID does not exist" containerID="fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2"
Sep 09 12:38:08.771423 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.771380    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2"} err="failed to get container status \"fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2\": rpc error: code = NotFound desc = could not find container \"fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2\": container with ID starting with fbd3a5d101d8cd732ddbe32d83748e693c89fce60fe99566e404011f15e887d2 not found: ID does not exist"
Sep 09 12:38:08.790473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.790445    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"]
Sep 09 12:38:08.796854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.796830    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1e223f-57b684cd6-j66dl"]
Sep 09 12:38:08.807449 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.807421    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"]
Sep 09 12:38:08.812210 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:08.812185    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1e223f-74ccc5b895-z579c"]
Sep 09 12:38:09.219153 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.219124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:09.319276 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.319239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:09.428292 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.428257    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"]
Sep 09 12:38:09.428548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.428521    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy" containerID="cri-o://b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4" gracePeriod=30
Sep 09 12:38:09.454997 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.454963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:09.872797 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.872762    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:09.895174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.895144    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"
Sep 09 12:38:09.947647 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.947614    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40acd1c2-94fb-4395-a126-75b9653ff07c-maas-api-service-ca\") pod \"40acd1c2-94fb-4395-a126-75b9653ff07c\" (UID: \"40acd1c2-94fb-4395-a126-75b9653ff07c\") "
Sep 09 12:38:09.947948 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:09.947916    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40acd1c2-94fb-4395-a126-75b9653ff07c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "40acd1c2-94fb-4395-a126-75b9653ff07c" (UID: "40acd1c2-94fb-4395-a126-75b9653ff07c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:10.049129 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:10.049036    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/40acd1c2-94fb-4395-a126-75b9653ff07c-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:10.102744 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:10.102711    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="641f1344-1baa-4b01-b74b-9f9f45c26d9c" path="/var/lib/kubelet/pods/641f1344-1baa-4b01-b74b-9f9f45c26d9c/volumes"
Sep 09 12:38:10.103071 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:10.103058    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6" path="/var/lib/kubelet/pods/712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6/volumes"
Sep 09 12:38:10.202754 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:10.202715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:10.463866 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:10.463835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:10.773477 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:10.773393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"
Sep 09 12:38:10.773477 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:10.773422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t" event={"ID":"40acd1c2-94fb-4395-a126-75b9653ff07c","Type":"ContainerDied","Data":"12ccf1ab68bfa3baec6aa368b9d55055a5bc7d58247590f0f645a4a0d7c93e34"}
Sep 09 12:38:10.773477 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:10.773468    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12ccf1ab68bfa3baec6aa368b9d55055a5bc7d58247590f0f645a4a0d7c93e34"
Sep 09 12:38:11.343286 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:11.343251    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:11.455127 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:11.455096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:11.542130 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:11.542094    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:14.675227 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.675204    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:38:14.687843 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.687818    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-data\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.687986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.687856    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-credential-socket\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.687986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.687879    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-certs\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.687986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.687898    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-token\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.687986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.687945    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjfmr\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-kube-api-access-mjfmr\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.688215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.687989    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-envoy\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.688215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688035    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istiod-ca-cert\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.688215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688059    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-podinfo\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.688215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688102    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-socket\") pod \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\" (UID: \"0f4f8c06-0d24-4e28-bf89-746be764dd6a\") "
Sep 09 12:38:14.688215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688135    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:14.688215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688138    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:14.688215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688172    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:14.688571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688356    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-data\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.688571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688376    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-credential-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.688571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688386    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:14.688571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688390    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-certs\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.688571 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.688481    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:14.690243 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.690216    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:14.690382 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.690264    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-kube-api-access-mjfmr" (OuterVolumeSpecName: "kube-api-access-mjfmr") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "kube-api-access-mjfmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:14.690564 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.690540    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:38:14.690675 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.690615    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f4f8c06-0d24-4e28-bf89-746be764dd6a" (UID: "0f4f8c06-0d24-4e28-bf89-746be764dd6a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:38:14.746954 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.746919    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:14.789159 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.789132    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-token\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.789159 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.789157    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjfmr\" (UniqueName: \"kubernetes.io/projected/0f4f8c06-0d24-4e28-bf89-746be764dd6a-kube-api-access-mjfmr\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.789362 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.789168    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-envoy\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.789362 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.789178    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istiod-ca-cert\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.789362 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.789186    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f4f8c06-0d24-4e28-bf89-746be764dd6a-istio-podinfo\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.789362 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.789194    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f4f8c06-0d24-4e28-bf89-746be764dd6a-workload-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:14.798927 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.798896    2579 generic.go:356] "Generic (PLEG): container finished" podID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerID="b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4" exitCode=0
Sep 09 12:38:14.799086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.798937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" event={"ID":"0f4f8c06-0d24-4e28-bf89-746be764dd6a","Type":"ContainerDied","Data":"b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4"}
Sep 09 12:38:14.799086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.798953    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"
Sep 09 12:38:14.799086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.799001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s" event={"ID":"0f4f8c06-0d24-4e28-bf89-746be764dd6a","Type":"ContainerDied","Data":"7dbb64044f91a306f691ea0e56f029930c770827496178c5900abd8502f684d4"}
Sep 09 12:38:14.799086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.799023    2579 scope.go:117] "RemoveContainer" containerID="b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4"
Sep 09 12:38:14.810544 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.810067    2579 scope.go:117] "RemoveContainer" containerID="a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27"
Sep 09 12:38:14.820281 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.820261    2579 scope.go:117] "RemoveContainer" containerID="b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4"
Sep 09 12:38:14.820588 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:38:14.820571    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4\": container with ID starting with b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4 not found: ID does not exist" containerID="b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4"
Sep 09 12:38:14.820659 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.820599    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4"} err="failed to get container status \"b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4\": rpc error: code = NotFound desc = could not find container \"b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4\": container with ID starting with b54a33a82b38f50e3591a22eeb57f56f980dee7355acbb4a2683066d06609be4 not found: ID does not exist"
Sep 09 12:38:14.820659 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.820616    2579 scope.go:117] "RemoveContainer" containerID="a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27"
Sep 09 12:38:14.820891 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:38:14.820872    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27\": container with ID starting with a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27 not found: ID does not exist" containerID="a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27"
Sep 09 12:38:14.820936 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.820894    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27"} err="failed to get container status \"a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27\": rpc error: code = NotFound desc = could not find container \"a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27\": container with ID starting with a37b6c16a2c92adc811b7eeb7b2caa83cf6327963e27ecdccd7f6432c3893d27 not found: ID does not exist"
Sep 09 12:38:14.825325 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.825294    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"]
Sep 09 12:38:14.829366 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.829341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:14.829657 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:14.829620    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-0f93fa-openshift-default-fd5bd98d8-mhm4s"]
Sep 09 12:38:15.008486 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.008457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:15.326605 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326526    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"]
Sep 09 12:38:15.326883 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326869    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641f1344-1baa-4b01-b74b-9f9f45c26d9c" containerName="payload-processing"
Sep 09 12:38:15.326932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326884    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="641f1344-1baa-4b01-b74b-9f9f45c26d9c" containerName="payload-processing"
Sep 09 12:38:15.326932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326892    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40acd1c2-94fb-4395-a126-75b9653ff07c" containerName="revoke-keys"
Sep 09 12:38:15.326932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326899    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="40acd1c2-94fb-4395-a126-75b9653ff07c" containerName="revoke-keys"
Sep 09 12:38:15.326932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326909    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6" containerName="payload-pre-processing"
Sep 09 12:38:15.326932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326914    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6" containerName="payload-pre-processing"
Sep 09 12:38:15.326932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326921    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy"
Sep 09 12:38:15.326932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326926    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy"
Sep 09 12:38:15.327137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326937    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy"
Sep 09 12:38:15.327137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326943    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy"
Sep 09 12:38:15.327137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.326994    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="712bd04a-7cab-4bc2-a4b8-4edcf4d9e1b6" containerName="payload-pre-processing"
Sep 09 12:38:15.327137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.327002    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="641f1344-1baa-4b01-b74b-9f9f45c26d9c" containerName="payload-processing"
Sep 09 12:38:15.327137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.327007    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="40acd1c2-94fb-4395-a126-75b9653ff07c" containerName="revoke-keys"
Sep 09 12:38:15.327137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.327013    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy"
Sep 09 12:38:15.327137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.327018    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" containerName="istio-proxy"
Sep 09 12:38:15.331409 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.331389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"
Sep 09 12:38:15.333586 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.333563    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:38:15.333810 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.333570    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:38:15.337437 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.337411    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"]
Sep 09 12:38:15.394114 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.394083    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03ec0c6-0e54-4fd4-92b4-f17da0d57c88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9\" (UID: \"e03ec0c6-0e54-4fd4-92b4-f17da0d57c88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"
Sep 09 12:38:15.494716 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.494681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03ec0c6-0e54-4fd4-92b4-f17da0d57c88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9\" (UID: \"e03ec0c6-0e54-4fd4-92b4-f17da0d57c88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"
Sep 09 12:38:15.495246 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.495224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03ec0c6-0e54-4fd4-92b4-f17da0d57c88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9\" (UID: \"e03ec0c6-0e54-4fd4-92b4-f17da0d57c88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"
Sep 09 12:38:15.582349 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.582245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:15.643371 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.643339    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"
Sep 09 12:38:15.917203 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:15.917156    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"]
Sep 09 12:38:15.919529 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:38:15.919501    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode03ec0c6_0e54_4fd4_92b4_f17da0d57c88.slice/crio-56cdf5e5eae760a903f3aa8e694e5caf566055cbcbbbb18718a79faf7f6e3d80 WatchSource:0}: Error finding container 56cdf5e5eae760a903f3aa8e694e5caf566055cbcbbbb18718a79faf7f6e3d80: Status 404 returned error can't find the container with id 56cdf5e5eae760a903f3aa8e694e5caf566055cbcbbbb18718a79faf7f6e3d80
Sep 09 12:38:16.102727 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.102695    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f4f8c06-0d24-4e28-bf89-746be764dd6a" path="/var/lib/kubelet/pods/0f4f8c06-0d24-4e28-bf89-746be764dd6a/volumes"
Sep 09 12:38:16.727418 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.727380    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:16.757430 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.757394    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"]
Sep 09 12:38:16.757683 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.757656    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns" podUID="31c87568-5b94-4d52-880c-f1af968ec7cb" containerName="payload-processing" containerID="cri-o://9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead" gracePeriod=30
Sep 09 12:38:16.763236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.763188    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"]
Sep 09 12:38:16.763432 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.763410    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh" podUID="0b688bbe-fe7f-4dbe-a158-e472f4247ed9" containerName="payload-pre-processing" containerID="cri-o://34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01" gracePeriod=30
Sep 09 12:38:16.800153 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.800118    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:16.810900 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.810869    2579 generic.go:356] "Generic (PLEG): container finished" podID="e03ec0c6-0e54-4fd4-92b4-f17da0d57c88" containerID="6109eab8f53dd4657a424fbaf61345dfa106c107076508fbc0b6847ae95e171c" exitCode=0
Sep 09 12:38:16.811070 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.810953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9" event={"ID":"e03ec0c6-0e54-4fd4-92b4-f17da0d57c88","Type":"ContainerDied","Data":"6109eab8f53dd4657a424fbaf61345dfa106c107076508fbc0b6847ae95e171c"}
Sep 09 12:38:16.811070 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.811041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9" event={"ID":"e03ec0c6-0e54-4fd4-92b4-f17da0d57c88","Type":"ContainerStarted","Data":"56cdf5e5eae760a903f3aa8e694e5caf566055cbcbbbb18718a79faf7f6e3d80"}
Sep 09 12:38:16.870492 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.870466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:16.937675 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.937622    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"]
Sep 09 12:38:16.939668 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.939614    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:16.940186 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:16.940166    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2475a5c7-wwc5t"]
Sep 09 12:38:17.005357 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.005322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:17.036811 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.036786    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:38:17.039149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.039130    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:38:17.107067 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.107035    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-plugins-config-volume\") pod \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\" (UID: \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\") "
Sep 09 12:38:17.107239 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.107114    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2g4n\" (UniqueName: \"kubernetes.io/projected/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-kube-api-access-z2g4n\") pod \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\" (UID: \"0b688bbe-fe7f-4dbe-a158-e472f4247ed9\") "
Sep 09 12:38:17.107239 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.107134    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31c87568-5b94-4d52-880c-f1af968ec7cb-plugins-config-volume\") pod \"31c87568-5b94-4d52-880c-f1af968ec7cb\" (UID: \"31c87568-5b94-4d52-880c-f1af968ec7cb\") "
Sep 09 12:38:17.107239 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.107165    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzkgg\" (UniqueName: \"kubernetes.io/projected/31c87568-5b94-4d52-880c-f1af968ec7cb-kube-api-access-jzkgg\") pod \"31c87568-5b94-4d52-880c-f1af968ec7cb\" (UID: \"31c87568-5b94-4d52-880c-f1af968ec7cb\") "
Sep 09 12:38:17.107524 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.107496    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31c87568-5b94-4d52-880c-f1af968ec7cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31c87568-5b94-4d52-880c-f1af968ec7cb" (UID: "31c87568-5b94-4d52-880c-f1af968ec7cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:17.107524 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.107496    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b688bbe-fe7f-4dbe-a158-e472f4247ed9" (UID: "0b688bbe-fe7f-4dbe-a158-e472f4247ed9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:17.109368 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.109341    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-kube-api-access-z2g4n" (OuterVolumeSpecName: "kube-api-access-z2g4n") pod "0b688bbe-fe7f-4dbe-a158-e472f4247ed9" (UID: "0b688bbe-fe7f-4dbe-a158-e472f4247ed9"). InnerVolumeSpecName "kube-api-access-z2g4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:17.109584 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.109568    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c87568-5b94-4d52-880c-f1af968ec7cb-kube-api-access-jzkgg" (OuterVolumeSpecName: "kube-api-access-jzkgg") pod "31c87568-5b94-4d52-880c-f1af968ec7cb" (UID: "31c87568-5b94-4d52-880c-f1af968ec7cb"). InnerVolumeSpecName "kube-api-access-jzkgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:17.208035 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.208000    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:17.208035 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.208033    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2g4n\" (UniqueName: \"kubernetes.io/projected/0b688bbe-fe7f-4dbe-a158-e472f4247ed9-kube-api-access-z2g4n\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:17.208228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.208044    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31c87568-5b94-4d52-880c-f1af968ec7cb-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:17.208228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.208052    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzkgg\" (UniqueName: \"kubernetes.io/projected/31c87568-5b94-4d52-880c-f1af968ec7cb-kube-api-access-jzkgg\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:17.295293 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.295254    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:17.817093 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.817055    2579 generic.go:356] "Generic (PLEG): container finished" podID="31c87568-5b94-4d52-880c-f1af968ec7cb" containerID="9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead" exitCode=0
Sep 09 12:38:17.817305 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.817097    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"
Sep 09 12:38:17.817305 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.817147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns" event={"ID":"31c87568-5b94-4d52-880c-f1af968ec7cb","Type":"ContainerDied","Data":"9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead"}
Sep 09 12:38:17.817305 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.817199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns" event={"ID":"31c87568-5b94-4d52-880c-f1af968ec7cb","Type":"ContainerDied","Data":"b194495d78ab4af71b0d0e203c74ab6179db5644051abf58ec5aee57cf7dcce0"}
Sep 09 12:38:17.817305 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.817227    2579 scope.go:117] "RemoveContainer" containerID="9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead"
Sep 09 12:38:17.818546 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.818474    2579 generic.go:356] "Generic (PLEG): container finished" podID="0b688bbe-fe7f-4dbe-a158-e472f4247ed9" containerID="34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01" exitCode=0
Sep 09 12:38:17.818546 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.818508    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"
Sep 09 12:38:17.818665 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.818546    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh" event={"ID":"0b688bbe-fe7f-4dbe-a158-e472f4247ed9","Type":"ContainerDied","Data":"34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01"}
Sep 09 12:38:17.818665 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.818600    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh" event={"ID":"0b688bbe-fe7f-4dbe-a158-e472f4247ed9","Type":"ContainerDied","Data":"5d4a0dffde07a7cab20cff949d6ad9ad8a2b236faf7c5f027b48d069fcca7d7d"}
Sep 09 12:38:17.827580 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.827545    2579 scope.go:117] "RemoveContainer" containerID="9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead"
Sep 09 12:38:17.827919 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:38:17.827895    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead\": container with ID starting with 9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead not found: ID does not exist" containerID="9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead"
Sep 09 12:38:17.828025 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.827928    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead"} err="failed to get container status \"9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead\": rpc error: code = NotFound desc = could not find container \"9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead\": container with ID starting with 9178d34245017a1877e99a50b114bd28f63e858d6f8e0d8ec126903e31b32ead not found: ID does not exist"
Sep 09 12:38:17.828025 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.827954    2579 scope.go:117] "RemoveContainer" containerID="34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01"
Sep 09 12:38:17.836233 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.836213    2579 scope.go:117] "RemoveContainer" containerID="34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01"
Sep 09 12:38:17.836524 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:38:17.836505    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01\": container with ID starting with 34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01 not found: ID does not exist" containerID="34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01"
Sep 09 12:38:17.836599 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.836529    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01"} err="failed to get container status \"34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01\": rpc error: code = NotFound desc = could not find container \"34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01\": container with ID starting with 34431dc1f0733b90146ce637c1f5df85368d9a00f410789670e0562f30ae7d01 not found: ID does not exist"
Sep 09 12:38:17.845024 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.844992    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"]
Sep 09 12:38:17.855520 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.855493    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2475a5c7-7f59d55d6-8d2fh"]
Sep 09 12:38:17.872945 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.872715    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"]
Sep 09 12:38:17.877381 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.877350    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2475a5c7-6dbc7ff7b9-5s9ns"]
Sep 09 12:38:17.951775 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:17.951752    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"
Sep 09 12:38:18.014736 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.014704    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03ec0c6-0e54-4fd4-92b4-f17da0d57c88-maas-api-service-ca\") pod \"e03ec0c6-0e54-4fd4-92b4-f17da0d57c88\" (UID: \"e03ec0c6-0e54-4fd4-92b4-f17da0d57c88\") "
Sep 09 12:38:18.015071 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.015046    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e03ec0c6-0e54-4fd4-92b4-f17da0d57c88-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e03ec0c6-0e54-4fd4-92b4-f17da0d57c88" (UID: "e03ec0c6-0e54-4fd4-92b4-f17da0d57c88"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:18.102129 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.102046    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b688bbe-fe7f-4dbe-a158-e472f4247ed9" path="/var/lib/kubelet/pods/0b688bbe-fe7f-4dbe-a158-e472f4247ed9/volumes"
Sep 09 12:38:18.102408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.102394    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31c87568-5b94-4d52-880c-f1af968ec7cb" path="/var/lib/kubelet/pods/31c87568-5b94-4d52-880c-f1af968ec7cb/volumes"
Sep 09 12:38:18.102694 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.102682    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40acd1c2-94fb-4395-a126-75b9653ff07c" path="/var/lib/kubelet/pods/40acd1c2-94fb-4395-a126-75b9653ff07c/volumes"
Sep 09 12:38:18.116046 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.116020    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e03ec0c6-0e54-4fd4-92b4-f17da0d57c88-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:18.827660 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.827622    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"
Sep 09 12:38:18.827831 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.827613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9" event={"ID":"e03ec0c6-0e54-4fd4-92b4-f17da0d57c88","Type":"ContainerDied","Data":"56cdf5e5eae760a903f3aa8e694e5caf566055cbcbbbb18718a79faf7f6e3d80"}
Sep 09 12:38:18.827831 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:18.827752    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56cdf5e5eae760a903f3aa8e694e5caf566055cbcbbbb18718a79faf7f6e3d80"
Sep 09 12:38:23.963269 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:23.963239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:24.029371 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:24.029335    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:24.669037 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:24.669003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:25.178853 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.178820    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:25.278321 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.278283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:25.378227 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.378185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:25.858760 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.858717    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:25.885476 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.885444    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"]
Sep 09 12:38:25.885743 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.885716    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv" podUID="a1801864-6611-4730-bb61-13931c0fbcc0" containerName="payload-processing" containerID="cri-o://0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34" gracePeriod=30
Sep 09 12:38:25.892969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.892944    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"]
Sep 09 12:38:25.893452 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.893158    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6" podUID="f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0" containerName="payload-pre-processing" containerID="cri-o://28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6" gracePeriod=30
Sep 09 12:38:25.959751 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:25.959626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:26.057496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.057454    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"]
Sep 09 12:38:26.059495 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.059462    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5e5923-p86x9"]
Sep 09 12:38:26.059858 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.059837    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:26.103831 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.103798    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e03ec0c6-0e54-4fd4-92b4-f17da0d57c88" path="/var/lib/kubelet/pods/e03ec0c6-0e54-4fd4-92b4-f17da0d57c88/volumes"
Sep 09 12:38:26.155220 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.155192    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:38:26.158049 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.158029    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:38:26.159886 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.159869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:26.281794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.281757    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1801864-6611-4730-bb61-13931c0fbcc0-plugins-config-volume\") pod \"a1801864-6611-4730-bb61-13931c0fbcc0\" (UID: \"a1801864-6611-4730-bb61-13931c0fbcc0\") "
Sep 09 12:38:26.281794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.281801    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-plugins-config-volume\") pod \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\" (UID: \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\") "
Sep 09 12:38:26.282304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.281906    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jt6mm\" (UniqueName: \"kubernetes.io/projected/a1801864-6611-4730-bb61-13931c0fbcc0-kube-api-access-jt6mm\") pod \"a1801864-6611-4730-bb61-13931c0fbcc0\" (UID: \"a1801864-6611-4730-bb61-13931c0fbcc0\") "
Sep 09 12:38:26.282304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.281954    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtxgk\" (UniqueName: \"kubernetes.io/projected/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-kube-api-access-rtxgk\") pod \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\" (UID: \"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0\") "
Sep 09 12:38:26.282304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.282211    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1801864-6611-4730-bb61-13931c0fbcc0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1801864-6611-4730-bb61-13931c0fbcc0" (UID: "a1801864-6611-4730-bb61-13931c0fbcc0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:26.282304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.282248    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0" (UID: "f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:38:26.283977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.283952    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1801864-6611-4730-bb61-13931c0fbcc0-kube-api-access-jt6mm" (OuterVolumeSpecName: "kube-api-access-jt6mm") pod "a1801864-6611-4730-bb61-13931c0fbcc0" (UID: "a1801864-6611-4730-bb61-13931c0fbcc0"). InnerVolumeSpecName "kube-api-access-jt6mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:26.284076 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.283952    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-kube-api-access-rtxgk" (OuterVolumeSpecName: "kube-api-access-rtxgk") pod "f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0" (UID: "f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0"). InnerVolumeSpecName "kube-api-access-rtxgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:38:26.383086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.383049    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtxgk\" (UniqueName: \"kubernetes.io/projected/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-kube-api-access-rtxgk\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:26.383086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.383079    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1801864-6611-4730-bb61-13931c0fbcc0-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:26.383086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.383091    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:26.383320 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.383099    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt6mm\" (UniqueName: \"kubernetes.io/projected/a1801864-6611-4730-bb61-13931c0fbcc0-kube-api-access-jt6mm\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:38:26.452046 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.452015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:26.866603 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.866508    2579 generic.go:356] "Generic (PLEG): container finished" podID="f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0" containerID="28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6" exitCode=0
Sep 09 12:38:26.866603 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.866554    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"
Sep 09 12:38:26.866801 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.866598    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6" event={"ID":"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0","Type":"ContainerDied","Data":"28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6"}
Sep 09 12:38:26.866801 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.866666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6" event={"ID":"f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0","Type":"ContainerDied","Data":"d3ad33a690c6217df85b4cc01f5feb9209d962e9217223454083996256e4e8a9"}
Sep 09 12:38:26.866801 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.866691    2579 scope.go:117] "RemoveContainer" containerID="28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6"
Sep 09 12:38:26.867897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.867865    2579 generic.go:356] "Generic (PLEG): container finished" podID="a1801864-6611-4730-bb61-13931c0fbcc0" containerID="0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34" exitCode=0
Sep 09 12:38:26.867987 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.867895    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv" event={"ID":"a1801864-6611-4730-bb61-13931c0fbcc0","Type":"ContainerDied","Data":"0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34"}
Sep 09 12:38:26.867987 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.867914    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"
Sep 09 12:38:26.867987 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.867938    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv" event={"ID":"a1801864-6611-4730-bb61-13931c0fbcc0","Type":"ContainerDied","Data":"753679d8c5fff648f64112bb2abb9149c1c0be8a51a93aa61b75d102ae37ac4c"}
Sep 09 12:38:26.877020 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.877003    2579 scope.go:117] "RemoveContainer" containerID="28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6"
Sep 09 12:38:26.877403 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:38:26.877377    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6\": container with ID starting with 28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6 not found: ID does not exist" containerID="28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6"
Sep 09 12:38:26.877472 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.877414    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6"} err="failed to get container status \"28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6\": rpc error: code = NotFound desc = could not find container \"28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6\": container with ID starting with 28e5628e319e58d3213230293c891b30da801a7197b7d03c40406c8cf9dc34c6 not found: ID does not exist"
Sep 09 12:38:26.877472 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.877431    2579 scope.go:117] "RemoveContainer" containerID="0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34"
Sep 09 12:38:26.885153 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.885137    2579 scope.go:117] "RemoveContainer" containerID="0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34"
Sep 09 12:38:26.885415 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:38:26.885392    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34\": container with ID starting with 0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34 not found: ID does not exist" containerID="0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34"
Sep 09 12:38:26.885495 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.885417    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34"} err="failed to get container status \"0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34\": rpc error: code = NotFound desc = could not find container \"0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34\": container with ID starting with 0286d32a9ae9fa91b3f9d41a6da8883ffcf3edce58f96bec98a6aa0cff928f34 not found: ID does not exist"
Sep 09 12:38:26.888762 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.888738    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"]
Sep 09 12:38:26.892394 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.892373    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5e5923-97498fbb6-8nhv6"]
Sep 09 12:38:26.902759 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.902737    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"]
Sep 09 12:38:26.906451 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:26.906431    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-5e5923-6877768669-wx2xv"]
Sep 09 12:38:28.103809 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:28.103776    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1801864-6611-4730-bb61-13931c0fbcc0" path="/var/lib/kubelet/pods/a1801864-6611-4730-bb61-13931c0fbcc0/volumes"
Sep 09 12:38:28.104172 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:28.104128    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0" path="/var/lib/kubelet/pods/f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0/volumes"
Sep 09 12:38:31.053511 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:31.053475    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:31.822415 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:31.822369    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:31.932306 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:31.932275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:32.032854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:32.032810    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:32.193369 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:32.193280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:33.224232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:33.224202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:33.288962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:33.288930    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:33.489892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:33.489803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:34.140573 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.140536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:34.397564 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.397480    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"]
Sep 09 12:38:34.397995 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.397957    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c87568-5b94-4d52-880c-f1af968ec7cb" containerName="payload-processing"
Sep 09 12:38:34.397995 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.397974    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c87568-5b94-4d52-880c-f1af968ec7cb" containerName="payload-processing"
Sep 09 12:38:34.397995 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.397987    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1801864-6611-4730-bb61-13931c0fbcc0" containerName="payload-processing"
Sep 09 12:38:34.397995 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.397996    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1801864-6611-4730-bb61-13931c0fbcc0" containerName="payload-processing"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398018    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0" containerName="payload-pre-processing"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398027    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0" containerName="payload-pre-processing"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398039    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b688bbe-fe7f-4dbe-a158-e472f4247ed9" containerName="payload-pre-processing"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398047    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b688bbe-fe7f-4dbe-a158-e472f4247ed9" containerName="payload-pre-processing"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398062    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e03ec0c6-0e54-4fd4-92b4-f17da0d57c88" containerName="revoke-keys"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398070    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e03ec0c6-0e54-4fd4-92b4-f17da0d57c88" containerName="revoke-keys"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398144    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e03ec0c6-0e54-4fd4-92b4-f17da0d57c88" containerName="revoke-keys"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398155    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1801864-6611-4730-bb61-13931c0fbcc0" containerName="payload-processing"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398165    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b688bbe-fe7f-4dbe-a158-e472f4247ed9" containerName="payload-pre-processing"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398176    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6665a5d-8e94-4a83-a6f0-fb4c4616e1e0" containerName="payload-pre-processing"
Sep 09 12:38:34.398184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.398186    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="31c87568-5b94-4d52-880c-f1af968ec7cb" containerName="payload-processing"
Sep 09 12:38:34.402863 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.402836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.404909 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.404886    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-283adb-openshift-default-dockercfg-7ssr7\""
Sep 09 12:38:34.417613 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.417565    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"]
Sep 09 12:38:34.442183 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.442142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:34.452959 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.452922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-socket\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.453101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.452969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1de361f-3185-4696-8908-0bcf54baf582-istiod-ca-cert\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.453101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.453009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-data\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.453101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.453045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-credential-socket\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.453101 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.453078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1de361f-3185-4696-8908-0bcf54baf582-istio-podinfo\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.453316 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.453106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-istio-token\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.453316 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.453135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prtc7\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-kube-api-access-prtc7\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.453316 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.453208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-certs\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.453316 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.453264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-envoy\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554351 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-istio-token\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554351 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prtc7\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-kube-api-access-prtc7\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-certs\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-envoy\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-socket\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1de361f-3185-4696-8908-0bcf54baf582-istiod-ca-cert\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-data\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-credential-socket\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554929 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1de361f-3185-4696-8908-0bcf54baf582-istio-podinfo\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.554988 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.554941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-certs\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.555044 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.555017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-credential-socket\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.555109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.555079    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-socket\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.555170 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.555149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-data\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.555429 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.555404    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1de361f-3185-4696-8908-0bcf54baf582-istiod-ca-cert\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.557032 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.557005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1de361f-3185-4696-8908-0bcf54baf582-istio-podinfo\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.557145 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.557086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-envoy\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.562719 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.562690    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-istio-token\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.562947 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.562927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prtc7\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-kube-api-access-prtc7\") pod \"e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") " pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:34.719340 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:34.719302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:35.004998 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:35.004911    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"]
Sep 09 12:38:35.009546 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:38:35.009510    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1de361f_3185_4696_8908_0bcf54baf582.slice/crio-691397cce8c773cdf6d14c965a7954e72ff8a09d905f982f1ec5fdb518434dab WatchSource:0}: Error finding container 691397cce8c773cdf6d14c965a7954e72ff8a09d905f982f1ec5fdb518434dab: Status 404 returned error can't find the container with id 691397cce8c773cdf6d14c965a7954e72ff8a09d905f982f1ec5fdb518434dab
Sep 09 12:38:35.011839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:35.011802    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:38:35.011928 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:35.011881    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:38:35.011928 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:35.011911    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:38:35.218688 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:35.218626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:35.916870 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:35.916836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" event={"ID":"c1de361f-3185-4696-8908-0bcf54baf582","Type":"ContainerStarted","Data":"0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d"}
Sep 09 12:38:35.917256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:35.916886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" event={"ID":"c1de361f-3185-4696-8908-0bcf54baf582","Type":"ContainerStarted","Data":"691397cce8c773cdf6d14c965a7954e72ff8a09d905f982f1ec5fdb518434dab"}
Sep 09 12:38:35.937938 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:35.937887    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" podStartSLOduration=1.9378701409999999 podStartE2EDuration="1.937870141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:35.936688432 +0000 UTC m=+1132.260908963" watchObservedRunningTime="2026-09-09 12:38:35.937870141 +0000 UTC m=+1132.262090671"
Sep 09 12:38:36.719778 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:36.719739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:36.725231 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:36.725203    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:36.922302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:36.922272    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:36.923156 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:36.923138    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:37.849159 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:37.849126    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:37.973697 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:37.973663    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:38.733572 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:38.733539    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:38.837137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:38.837107    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:39.005951 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.005862    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:39.106292 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.106259    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:39.206456 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.206417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:39.561145 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.561103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:39.654617 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.654566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:39.655507 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.655486    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"]
Sep 09 12:38:39.658851 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.658833    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:39.660802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.660780    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-283adb\""
Sep 09 12:38:39.669234 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.669214    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"]
Sep 09 12:38:39.732029 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.731990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:39.802629 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.802592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4p76\" (UniqueName: \"kubernetes.io/projected/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-kube-api-access-v4p76\") pod \"payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2\" (UID: \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:39.802629 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.802647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2\" (UID: \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:39.903149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.903070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4p76\" (UniqueName: \"kubernetes.io/projected/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-kube-api-access-v4p76\") pod \"payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2\" (UID: \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:39.903149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.903111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2\" (UID: \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:39.903839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.903818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2\" (UID: \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:39.915944 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.915911    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4p76\" (UniqueName: \"kubernetes.io/projected/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-kube-api-access-v4p76\") pod \"payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2\" (UID: \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:39.968965 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:39.968929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:40.000187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:40.000144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:40.245677 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:40.245653    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"]
Sep 09 12:38:40.248393 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:38:40.248365    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb19cc84_83dd_45ed_94c0_8fb4edb69ec5.slice/crio-f7a9998fc9c37b288f348737e5d330706ab7a449a79975756e318f44be7fda3d WatchSource:0}: Error finding container f7a9998fc9c37b288f348737e5d330706ab7a449a79975756e318f44be7fda3d: Status 404 returned error can't find the container with id f7a9998fc9c37b288f348737e5d330706ab7a449a79975756e318f44be7fda3d
Sep 09 12:38:40.307140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:40.307106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:40.943047 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:40.943004    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2" event={"ID":"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5","Type":"ContainerStarted","Data":"2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167"}
Sep 09 12:38:40.943230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:40.943066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2" event={"ID":"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5","Type":"ContainerStarted","Data":"f7a9998fc9c37b288f348737e5d330706ab7a449a79975756e318f44be7fda3d"}
Sep 09 12:38:40.943230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:40.943130    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:40.972625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:40.972576    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2" podStartSLOduration=1.972561764 podStartE2EDuration="1.972561764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:40.971132918 +0000 UTC m=+1137.295353449" watchObservedRunningTime="2026-09-09 12:38:40.972561764 +0000 UTC m=+1137.296782295"
Sep 09 12:38:42.933763 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:42.933722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:43.078201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:43.078148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:43.192671 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:43.192571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:43.292198 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:43.292157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:43.430362 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:43.430329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:43.532741 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:43.532703    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:44.481603 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:44.481563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:44.614017 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:44.613979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:44.874569 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:44.874478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:45.004523 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.004486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:45.104260 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.104220    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:45.128947 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.128876    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"]
Sep 09 12:38:45.133870 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.133851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.135932 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.135908    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-a3f43f-dockercfg-cptxz\""
Sep 09 12:38:45.136071 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.135993    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-a3f43f\""
Sep 09 12:38:45.141714 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.141687    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"]
Sep 09 12:38:45.253713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.253659    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk2nr\" (UniqueName: \"kubernetes.io/projected/bc4360ca-8972-4224-aedf-a7ee36202f36-kube-api-access-sk2nr\") pod \"payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2\" (UID: \"bc4360ca-8972-4224-aedf-a7ee36202f36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.253928 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.253769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4360ca-8972-4224-aedf-a7ee36202f36-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2\" (UID: \"bc4360ca-8972-4224-aedf-a7ee36202f36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.355044 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.354994    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk2nr\" (UniqueName: \"kubernetes.io/projected/bc4360ca-8972-4224-aedf-a7ee36202f36-kube-api-access-sk2nr\") pod \"payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2\" (UID: \"bc4360ca-8972-4224-aedf-a7ee36202f36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.355242 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.355097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4360ca-8972-4224-aedf-a7ee36202f36-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2\" (UID: \"bc4360ca-8972-4224-aedf-a7ee36202f36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.356086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.356053    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4360ca-8972-4224-aedf-a7ee36202f36-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2\" (UID: \"bc4360ca-8972-4224-aedf-a7ee36202f36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.365341 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.365311    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk2nr\" (UniqueName: \"kubernetes.io/projected/bc4360ca-8972-4224-aedf-a7ee36202f36-kube-api-access-sk2nr\") pod \"payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2\" (UID: \"bc4360ca-8972-4224-aedf-a7ee36202f36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.445458 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.445377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.726496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.726472    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"]
Sep 09 12:38:45.728587 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:38:45.728563    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4360ca_8972_4224_aedf_a7ee36202f36.slice/crio-3dfd4cf2e1fe092e46584f1075b248c542fd2918fd24ed48baa4bc1ed76cdd24 WatchSource:0}: Error finding container 3dfd4cf2e1fe092e46584f1075b248c542fd2918fd24ed48baa4bc1ed76cdd24: Status 404 returned error can't find the container with id 3dfd4cf2e1fe092e46584f1075b248c542fd2918fd24ed48baa4bc1ed76cdd24
Sep 09 12:38:45.973191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.973141    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2" event={"ID":"bc4360ca-8972-4224-aedf-a7ee36202f36","Type":"ContainerStarted","Data":"aaed71e59243c45d3525319172da47eb75fd1a88cfba6f71b205156c5233dcea"}
Sep 09 12:38:45.973404 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.973217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2" event={"ID":"bc4360ca-8972-4224-aedf-a7ee36202f36","Type":"ContainerStarted","Data":"3dfd4cf2e1fe092e46584f1075b248c542fd2918fd24ed48baa4bc1ed76cdd24"}
Sep 09 12:38:45.974046 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.974017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:45.992363 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:45.992273    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2" podStartSLOduration=0.992258463 podStartE2EDuration="992.258463ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:38:45.989945044 +0000 UTC m=+1142.314165577" watchObservedRunningTime="2026-09-09 12:38:45.992258463 +0000 UTC m=+1142.316478993"
Sep 09 12:38:48.551779 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:48.551741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:51.950245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:51.950212    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:38:52.640744 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:52.640711    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:35158->10.134.0.45:15021: read: connection reset by peer" start-of-body=
Sep 09 12:38:52.640892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:52.640786    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:35158->10.134.0.45:15021: read: connection reset by peer"
Sep 09 12:38:53.019903 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:53.019867    2579 generic.go:356] "Generic (PLEG): container finished" podID="c1de361f-3185-4696-8908-0bcf54baf582" containerID="0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d" exitCode=0
Sep 09 12:38:53.020381 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:53.019919    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" event={"ID":"c1de361f-3185-4696-8908-0bcf54baf582","Type":"ContainerDied","Data":"0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d"}
Sep 09 12:38:53.020452 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:53.020387    2579 scope.go:117] "RemoveContainer" containerID="0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d"
Sep 09 12:38:53.021470 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:53.021441    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:38:53.021544 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:53.021529    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:38:53.021582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:53.021568    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:38:54.026630 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:54.026597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" event={"ID":"c1de361f-3185-4696-8908-0bcf54baf582","Type":"ContainerStarted","Data":"88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2"}
Sep 09 12:38:54.719760 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:54.719715    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:54.720088 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:54.720066    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 09 12:38:54.720160 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:54.720110    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 09 12:38:55.720149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:55.720110    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 09 12:38:55.720554 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:55.720169    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 09 12:38:56.218752 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:56.218723    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:38:56.720188 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:56.720155    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 09 12:38:56.720564 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:56.720213    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 09 12:38:57.720660 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:57.720607    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 09 12:38:57.721051 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:57.720689    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 09 12:38:57.985939 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:57.985851    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:38:58.720714 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:58.720672    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 09 12:38:58.721122 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:58.720731    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 09 12:38:59.723431 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:59.723393    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:59.723877 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:59.723684    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:38:59.724401 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:38:59.724383    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:39:01.174775 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:01.174737    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:02.840754 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:02.840721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:03.173420 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:03.173327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:03.483121 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:03.483092    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:03.912878 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:03.912799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:05.561256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:05.561222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:06.397122 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:06.397045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:06.497795 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:06.497761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:07.664221 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:07.664177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:07.765097 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:07.765058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:07.865273 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:07.865233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:08.153256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.153208    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:08.228250 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.228214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:08.254432 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.254401    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"]
Sep 09 12:39:08.258017 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.257988    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:08.260393 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.260368    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-fe6606\""
Sep 09 12:39:08.261415 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.261397    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-fe6606-dockercfg-7gp5r\""
Sep 09 12:39:08.271835 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.271810    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"]
Sep 09 12:39:08.353878 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.353843    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff11232-42ca-4288-a4dd-b99b124b144c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8\" (UID: \"fff11232-42ca-4288-a4dd-b99b124b144c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:08.353878 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.353881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9kkq\" (UniqueName: \"kubernetes.io/projected/fff11232-42ca-4288-a4dd-b99b124b144c-kube-api-access-x9kkq\") pod \"payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8\" (UID: \"fff11232-42ca-4288-a4dd-b99b124b144c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:08.455190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.455153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff11232-42ca-4288-a4dd-b99b124b144c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8\" (UID: \"fff11232-42ca-4288-a4dd-b99b124b144c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:08.455385 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.455195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9kkq\" (UniqueName: \"kubernetes.io/projected/fff11232-42ca-4288-a4dd-b99b124b144c-kube-api-access-x9kkq\") pod \"payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8\" (UID: \"fff11232-42ca-4288-a4dd-b99b124b144c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:08.455933 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.455912    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff11232-42ca-4288-a4dd-b99b124b144c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8\" (UID: \"fff11232-42ca-4288-a4dd-b99b124b144c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:08.463162 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.463138    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9kkq\" (UniqueName: \"kubernetes.io/projected/fff11232-42ca-4288-a4dd-b99b124b144c-kube-api-access-x9kkq\") pod \"payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8\" (UID: \"fff11232-42ca-4288-a4dd-b99b124b144c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:08.557513 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.557477    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:08.567336 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.567308    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:08.842499 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.842369    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"]
Sep 09 12:39:08.849666 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.849613    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"]
Sep 09 12:39:08.852406 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:39:08.852362    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff11232_42ca_4288_a4dd_b99b124b144c.slice/crio-00019a33811c117b86d00f51f587229fe3bd26b130edccde7459e25ead5ba558 WatchSource:0}: Error finding container 00019a33811c117b86d00f51f587229fe3bd26b130edccde7459e25ead5ba558: Status 404 returned error can't find the container with id 00019a33811c117b86d00f51f587229fe3bd26b130edccde7459e25ead5ba558
Sep 09 12:39:08.854539 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.854518    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.856800 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.856781    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-590823-openshift-default-dockercfg-6cxnz\""
Sep 09 12:39:08.864971 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.864897    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"]
Sep 09 12:39:08.876517 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.876492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:08.959214 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-socket\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.959214 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-token\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.959453 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-data\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.959453 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959249    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-credential-socket\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.959453 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959265    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-podinfo\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.959453 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-envoy\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.959453 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9tfd\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-kube-api-access-r9tfd\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.959686 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istiod-ca-cert\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.959686 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.959524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-certs\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:08.974833 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:08.974768    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:09.061112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061066    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9tfd\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-kube-api-access-r9tfd\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.061304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istiod-ca-cert\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.061304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-certs\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.061304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-socket\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.061304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-token\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.061304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061278    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-data\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.061560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-credential-socket\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.061560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-podinfo\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.061560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.061381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-envoy\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.062173 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.062143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-socket\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.062563 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.062522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-credential-socket\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.062846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.062741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istiod-ca-cert\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.062846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.062811    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-certs\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.063246 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.063231    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-data\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.064840 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.064816    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-envoy\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.065133 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.065115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-podinfo\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.070107 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.070083    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9tfd\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-kube-api-access-r9tfd\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.070384 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.070364    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-token\") pod \"e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") " pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.110059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.109987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8" event={"ID":"fff11232-42ca-4288-a4dd-b99b124b144c","Type":"ContainerStarted","Data":"887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea"}
Sep 09 12:39:09.110059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.110036    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8" event={"ID":"fff11232-42ca-4288-a4dd-b99b124b144c","Type":"ContainerStarted","Data":"00019a33811c117b86d00f51f587229fe3bd26b130edccde7459e25ead5ba558"}
Sep 09 12:39:09.110245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.110082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:09.127752 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.127708    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8" podStartSLOduration=1.127693536 podStartE2EDuration="1.127693536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:39:09.125964508 +0000 UTC m=+1165.450185044" watchObservedRunningTime="2026-09-09 12:39:09.127693536 +0000 UTC m=+1165.451914067"
Sep 09 12:39:09.182789 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.182759    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:09.456447 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.456420    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"]
Sep 09 12:39:09.457429 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:39:09.457402    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b633ad_9f34_4346_baa1_e3ec33ef7c30.slice/crio-f3583e2f2dd665c164c88660ef29dd760694fc3aa3ab6a846d11bbd98aa5dd76 WatchSource:0}: Error finding container f3583e2f2dd665c164c88660ef29dd760694fc3aa3ab6a846d11bbd98aa5dd76: Status 404 returned error can't find the container with id f3583e2f2dd665c164c88660ef29dd760694fc3aa3ab6a846d11bbd98aa5dd76
Sep 09 12:39:09.459741 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.459711    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:39:09.459802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.459775    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:39:09.459840 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:09.459802    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:39:10.116126 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:10.116081    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" event={"ID":"33b633ad-9f34-4346-baa1-e3ec33ef7c30","Type":"ContainerStarted","Data":"a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263"}
Sep 09 12:39:10.116622 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:10.116147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" event={"ID":"33b633ad-9f34-4346-baa1-e3ec33ef7c30","Type":"ContainerStarted","Data":"f3583e2f2dd665c164c88660ef29dd760694fc3aa3ab6a846d11bbd98aa5dd76"}
Sep 09 12:39:10.136527 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:10.136463    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" podStartSLOduration=2.136441033 podStartE2EDuration="2.136441033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:39:10.135305698 +0000 UTC m=+1166.459526228" watchObservedRunningTime="2026-09-09 12:39:10.136441033 +0000 UTC m=+1166.460661564"
Sep 09 12:39:10.183568 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:10.183535    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:10.188730 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:10.188703    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:10.226985 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:10.226950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:10.303791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:10.303760    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:11.121711 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:11.121677    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:11.122913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:11.122888    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:12.217862 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:12.217829    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:12.321097 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:12.321062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:13.322850 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:13.322818    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:13.423976 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:13.423933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:13.685190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:13.685099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:13.853998 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:13.853940    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:13.951018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:13.950987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:14.133617 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.133574    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"]
Sep 09 12:39:14.137495 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.137472    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:14.139558 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.139537    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-590823-dockercfg-xtfnd\""
Sep 09 12:39:14.139721 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.139540    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-590823\""
Sep 09 12:39:14.148862 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.148838    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"]
Sep 09 12:39:14.208975 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.208884    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8896c3c9-0832-48e8-8f16-b5354633862a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl\" (UID: \"8896c3c9-0832-48e8-8f16-b5354633862a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:14.208975 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.208936    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbkd8\" (UniqueName: \"kubernetes.io/projected/8896c3c9-0832-48e8-8f16-b5354633862a-kube-api-access-sbkd8\") pod \"payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl\" (UID: \"8896c3c9-0832-48e8-8f16-b5354633862a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:14.310254 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.310223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbkd8\" (UniqueName: \"kubernetes.io/projected/8896c3c9-0832-48e8-8f16-b5354633862a-kube-api-access-sbkd8\") pod \"payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl\" (UID: \"8896c3c9-0832-48e8-8f16-b5354633862a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:14.310422 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.310307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8896c3c9-0832-48e8-8f16-b5354633862a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl\" (UID: \"8896c3c9-0832-48e8-8f16-b5354633862a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:14.310934 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.310916    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8896c3c9-0832-48e8-8f16-b5354633862a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl\" (UID: \"8896c3c9-0832-48e8-8f16-b5354633862a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:14.318890 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.318862    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbkd8\" (UniqueName: \"kubernetes.io/projected/8896c3c9-0832-48e8-8f16-b5354633862a-kube-api-access-sbkd8\") pod \"payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl\" (UID: \"8896c3c9-0832-48e8-8f16-b5354633862a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:14.452560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.452523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:14.728297 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:14.728220    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"]
Sep 09 12:39:14.731533 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:39:14.731506    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8896c3c9_0832_48e8_8f16_b5354633862a.slice/crio-a1a22df76a8141c3f91eeaf7fda128f3063b999bfb6c5fdda5eb417d0455f53f WatchSource:0}: Error finding container a1a22df76a8141c3f91eeaf7fda128f3063b999bfb6c5fdda5eb417d0455f53f: Status 404 returned error can't find the container with id a1a22df76a8141c3f91eeaf7fda128f3063b999bfb6c5fdda5eb417d0455f53f
Sep 09 12:39:15.143892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:15.143804    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl" event={"ID":"8896c3c9-0832-48e8-8f16-b5354633862a","Type":"ContainerStarted","Data":"19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e"}
Sep 09 12:39:15.143892 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:15.143857    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl" event={"ID":"8896c3c9-0832-48e8-8f16-b5354633862a","Type":"ContainerStarted","Data":"a1a22df76a8141c3f91eeaf7fda128f3063b999bfb6c5fdda5eb417d0455f53f"}
Sep 09 12:39:15.144078 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:15.143968    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:15.160305 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:15.160252    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl" podStartSLOduration=1.160236395 podStartE2EDuration="1.160236395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:39:15.158901596 +0000 UTC m=+1171.483122128" watchObservedRunningTime="2026-09-09 12:39:15.160236395 +0000 UTC m=+1171.484456926"
Sep 09 12:39:20.117809 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:20.117777    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:39:25.567534 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:25.567484    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:25.961604 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:25.961573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:26.060448 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:26.060412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:26.150853 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:26.150815    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:39:28.595109 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:28.595073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:31.329466 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:31.329431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:32.229058 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:32.229021    2579 generic.go:356] "Generic (PLEG): container finished" podID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerID="a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263" exitCode=0
Sep 09 12:39:32.229263 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:32.229094    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" event={"ID":"33b633ad-9f34-4346-baa1-e3ec33ef7c30","Type":"ContainerDied","Data":"a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263"}
Sep 09 12:39:32.229510 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:32.229496    2579 scope.go:117] "RemoveContainer" containerID="a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263"
Sep 09 12:39:32.230593 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:32.230569    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:39:32.230660 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:32.230651    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:39:32.230702 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:32.230684    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:39:33.236259 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:33.236227    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" event={"ID":"33b633ad-9f34-4346-baa1-e3ec33ef7c30","Type":"ContainerStarted","Data":"69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915"}
Sep 09 12:39:34.183076 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:34.183034    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:34.183331 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:34.183289    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 12:39:34.183380 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:34.183332    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 12:39:35.184127 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:35.184088    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 12:39:35.184531 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:35.184145    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 12:39:35.337462 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:35.337426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:36.183740 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:36.183698    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 12:39:36.183941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:36.183778    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 12:39:37.082731 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:37.082694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:37.183980 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:37.183940    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 12:39:37.184164 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:37.184001    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 12:39:37.960187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:37.960153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:38.183952 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:38.183917    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 09 12:39:38.184323 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:38.183979    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 09 12:39:39.187496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:39.187459    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:39.187928 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:39.187743    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:39.188512 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:39.188491    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:39:46.563503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:46.563469    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:46.873516 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:46.873418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:47.206394 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:47.206360    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:49.615287 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:49.615250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:49.711568 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:49.711534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:50.877384 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:50.877351    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:50.977940 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:50.977907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:51.083306 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.083270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:51.375377 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.375346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:51.476915 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.476876    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:39:51.478883 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.478854    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"]
Sep 09 12:39:51.481282 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.481265    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:51.483249 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.483229    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6a4af97c\""
Sep 09 12:39:51.490396 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.490371    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"]
Sep 09 12:39:51.638450 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.638353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aca109c5-e333-4462-aafc-30b8c7278b04-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5\" (UID: \"aca109c5-e333-4462-aafc-30b8c7278b04\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:51.638450 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.638424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m8tf\" (UniqueName: \"kubernetes.io/projected/aca109c5-e333-4462-aafc-30b8c7278b04-kube-api-access-6m8tf\") pod \"payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5\" (UID: \"aca109c5-e333-4462-aafc-30b8c7278b04\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:51.739007 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.738951    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aca109c5-e333-4462-aafc-30b8c7278b04-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5\" (UID: \"aca109c5-e333-4462-aafc-30b8c7278b04\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:51.739195 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.739033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m8tf\" (UniqueName: \"kubernetes.io/projected/aca109c5-e333-4462-aafc-30b8c7278b04-kube-api-access-6m8tf\") pod \"payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5\" (UID: \"aca109c5-e333-4462-aafc-30b8c7278b04\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:51.739697 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.739672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aca109c5-e333-4462-aafc-30b8c7278b04-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5\" (UID: \"aca109c5-e333-4462-aafc-30b8c7278b04\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:51.748426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.748398    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m8tf\" (UniqueName: \"kubernetes.io/projected/aca109c5-e333-4462-aafc-30b8c7278b04-kube-api-access-6m8tf\") pod \"payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5\" (UID: \"aca109c5-e333-4462-aafc-30b8c7278b04\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:51.792404 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:51.792362    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:52.063883 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:52.063858    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"]
Sep 09 12:39:52.065877 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:39:52.065845    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca109c5_e333_4462_aafc_30b8c7278b04.slice/crio-03099ae2aad577764bb652fc8cf78f3e73b36640a3aa5d59e7f854644304be3d WatchSource:0}: Error finding container 03099ae2aad577764bb652fc8cf78f3e73b36640a3aa5d59e7f854644304be3d: Status 404 returned error can't find the container with id 03099ae2aad577764bb652fc8cf78f3e73b36640a3aa5d59e7f854644304be3d
Sep 09 12:39:52.347741 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:52.347653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5" event={"ID":"aca109c5-e333-4462-aafc-30b8c7278b04","Type":"ContainerStarted","Data":"d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0"}
Sep 09 12:39:52.347741 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:52.347715    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5" event={"ID":"aca109c5-e333-4462-aafc-30b8c7278b04","Type":"ContainerStarted","Data":"03099ae2aad577764bb652fc8cf78f3e73b36640a3aa5d59e7f854644304be3d"}
Sep 09 12:39:52.347941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:52.347770    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:39:52.365077 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:39:52.365025    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5" podStartSLOduration=1.365010731 podStartE2EDuration="1.365010731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:39:52.363371251 +0000 UTC m=+1208.687591783" watchObservedRunningTime="2026-09-09 12:39:52.365010731 +0000 UTC m=+1208.689231262"
Sep 09 12:40:03.354735 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:40:03.354704    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:40:14.611896 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:40:14.611860    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:40:28.297540 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:40:28.297505    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:40:34.782092 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:40:34.782057    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:40:44.181679 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:40:44.181649    2579 scope.go:117] "RemoveContainer" containerID="90b9c32d31e17205a7534b6d4cf0641b49901bacd92870ff5a8256cd72cdc045"
Sep 09 12:40:44.189751 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:40:44.189735    2579 scope.go:117] "RemoveContainer" containerID="de87abdf56d63aac5dbdccd50a4faf9623c64e678cbc9c8e0105dd0e9ef6465e"
Sep 09 12:40:50.427652 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:40:50.427602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:41:24.906997 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:24.906958    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"]
Sep 09 12:41:24.909066 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:24.909051    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"
Sep 09 12:41:24.911269 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:24.911248    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:41:24.911408 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:24.911290    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:41:24.917576 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:24.917550    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"]
Sep 09 12:41:25.026031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.025997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4c4d2e-6833-4a1a-9b41-917559799793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc\" (UID: \"7e4c4d2e-6833-4a1a-9b41-917559799793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"
Sep 09 12:41:25.056120 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.056089    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:41:25.127952 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.127192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4c4d2e-6833-4a1a-9b41-917559799793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc\" (UID: \"7e4c4d2e-6833-4a1a-9b41-917559799793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"
Sep 09 12:41:25.128140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.128090    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4c4d2e-6833-4a1a-9b41-917559799793-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc\" (UID: \"7e4c4d2e-6833-4a1a-9b41-917559799793\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"
Sep 09 12:41:25.220072 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.220044    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"
Sep 09 12:41:25.491055 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.491024    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"]
Sep 09 12:41:25.492819 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:41:25.492782    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4c4d2e_6833_4a1a_9b41_917559799793.slice/crio-fa5ab129a0f8dfd73f0c4ad1f1b85fe6294043686acd7ab2782da7bf9538c6f8 WatchSource:0}: Error finding container fa5ab129a0f8dfd73f0c4ad1f1b85fe6294043686acd7ab2782da7bf9538c6f8: Status 404 returned error can't find the container with id fa5ab129a0f8dfd73f0c4ad1f1b85fe6294043686acd7ab2782da7bf9538c6f8
Sep 09 12:41:25.812270 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.812182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc" event={"ID":"7e4c4d2e-6833-4a1a-9b41-917559799793","Type":"ContainerStarted","Data":"ddac718786a19ba5c95a1bcb2ea10d08f637bebd2866177d4320623f4f5bb86d"}
Sep 09 12:41:25.812270 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.812235    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc" event={"ID":"7e4c4d2e-6833-4a1a-9b41-917559799793","Type":"ContainerStarted","Data":"fa5ab129a0f8dfd73f0c4ad1f1b85fe6294043686acd7ab2782da7bf9538c6f8"}
Sep 09 12:41:25.828362 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:25.828312    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc" podStartSLOduration=1.828296778 podStartE2EDuration="1.828296778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:41:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:41:25.826489102 +0000 UTC m=+1302.150709624" watchObservedRunningTime="2026-09-09 12:41:25.828296778 +0000 UTC m=+1302.152517308"
Sep 09 12:41:26.819201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:26.819169    2579 generic.go:356] "Generic (PLEG): container finished" podID="7e4c4d2e-6833-4a1a-9b41-917559799793" containerID="ddac718786a19ba5c95a1bcb2ea10d08f637bebd2866177d4320623f4f5bb86d" exitCode=0
Sep 09 12:41:26.819591 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:26.819254    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc" event={"ID":"7e4c4d2e-6833-4a1a-9b41-917559799793","Type":"ContainerDied","Data":"ddac718786a19ba5c95a1bcb2ea10d08f637bebd2866177d4320623f4f5bb86d"}
Sep 09 12:41:27.953166 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:27.953142    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"
Sep 09 12:41:28.050927 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:28.050894    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4c4d2e-6833-4a1a-9b41-917559799793-maas-api-service-ca\") pod \"7e4c4d2e-6833-4a1a-9b41-917559799793\" (UID: \"7e4c4d2e-6833-4a1a-9b41-917559799793\") "
Sep 09 12:41:28.051290 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:28.051264    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e4c4d2e-6833-4a1a-9b41-917559799793-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7e4c4d2e-6833-4a1a-9b41-917559799793" (UID: "7e4c4d2e-6833-4a1a-9b41-917559799793"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:41:28.151576 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:28.151525    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e4c4d2e-6833-4a1a-9b41-917559799793-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:41:28.831052 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:28.831024    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"
Sep 09 12:41:28.831227 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:28.831066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc" event={"ID":"7e4c4d2e-6833-4a1a-9b41-917559799793","Type":"ContainerDied","Data":"fa5ab129a0f8dfd73f0c4ad1f1b85fe6294043686acd7ab2782da7bf9538c6f8"}
Sep 09 12:41:28.831227 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:28.831135    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa5ab129a0f8dfd73f0c4ad1f1b85fe6294043686acd7ab2782da7bf9538c6f8"
Sep 09 12:41:40.562400 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.562366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:41:40.594599 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.594561    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"]
Sep 09 12:41:40.594854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.594828    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5" podUID="aca109c5-e333-4462-aafc-30b8c7278b04" containerName="payload-pre-processing" containerID="cri-o://d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0" gracePeriod=30
Sep 09 12:41:40.631538 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.631491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:41:40.733604 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.733568    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:41:40.785801 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.785768    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"]
Sep 09 12:41:40.788389 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.788366    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a4af97c-m9fxc"]
Sep 09 12:41:40.791172 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.791138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:41:40.852388 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.852356    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:41:40.890854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.890819    2579 generic.go:356] "Generic (PLEG): container finished" podID="aca109c5-e333-4462-aafc-30b8c7278b04" containerID="d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0" exitCode=0
Sep 09 12:41:40.891018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.890882    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"
Sep 09 12:41:40.891018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.890883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5" event={"ID":"aca109c5-e333-4462-aafc-30b8c7278b04","Type":"ContainerDied","Data":"d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0"}
Sep 09 12:41:40.891018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.891004    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5" event={"ID":"aca109c5-e333-4462-aafc-30b8c7278b04","Type":"ContainerDied","Data":"03099ae2aad577764bb652fc8cf78f3e73b36640a3aa5d59e7f854644304be3d"}
Sep 09 12:41:40.891147 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.891027    2579 scope.go:117] "RemoveContainer" containerID="d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0"
Sep 09 12:41:40.901060 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.901039    2579 scope.go:117] "RemoveContainer" containerID="d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0"
Sep 09 12:41:40.901444 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:41:40.901414    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0\": container with ID starting with d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0 not found: ID does not exist" containerID="d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0"
Sep 09 12:41:40.901528 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.901454    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0"} err="failed to get container status \"d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0\": rpc error: code = NotFound desc = could not find container \"d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0\": container with ID starting with d8ff870511d2b312863c53567b0220e9b2e93435b3c8b52ac704088902f48fe0 not found: ID does not exist"
Sep 09 12:41:40.974082 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.974044    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aca109c5-e333-4462-aafc-30b8c7278b04-plugins-config-volume\") pod \"aca109c5-e333-4462-aafc-30b8c7278b04\" (UID: \"aca109c5-e333-4462-aafc-30b8c7278b04\") "
Sep 09 12:41:40.974278 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.974223    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m8tf\" (UniqueName: \"kubernetes.io/projected/aca109c5-e333-4462-aafc-30b8c7278b04-kube-api-access-6m8tf\") pod \"aca109c5-e333-4462-aafc-30b8c7278b04\" (UID: \"aca109c5-e333-4462-aafc-30b8c7278b04\") "
Sep 09 12:41:40.974439 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.974414    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aca109c5-e333-4462-aafc-30b8c7278b04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aca109c5-e333-4462-aafc-30b8c7278b04" (UID: "aca109c5-e333-4462-aafc-30b8c7278b04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:41:40.976284 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:40.976263    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aca109c5-e333-4462-aafc-30b8c7278b04-kube-api-access-6m8tf" (OuterVolumeSpecName: "kube-api-access-6m8tf") pod "aca109c5-e333-4462-aafc-30b8c7278b04" (UID: "aca109c5-e333-4462-aafc-30b8c7278b04"). InnerVolumeSpecName "kube-api-access-6m8tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:41:41.074930 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:41.074834    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m8tf\" (UniqueName: \"kubernetes.io/projected/aca109c5-e333-4462-aafc-30b8c7278b04-kube-api-access-6m8tf\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:41:41.074930 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:41.074867    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aca109c5-e333-4462-aafc-30b8c7278b04-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:41:41.219138 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:41.213542    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"]
Sep 09 12:41:41.219138 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:41.218613    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6a4af97c-65b5566cd6-k8wz5"]
Sep 09 12:41:42.102738 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:42.102703    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4c4d2e-6833-4a1a-9b41-917559799793" path="/var/lib/kubelet/pods/7e4c4d2e-6833-4a1a-9b41-917559799793/volumes"
Sep 09 12:41:42.103194 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:42.103071    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aca109c5-e333-4462-aafc-30b8c7278b04" path="/var/lib/kubelet/pods/aca109c5-e333-4462-aafc-30b8c7278b04/volumes"
Sep 09 12:41:44.211012 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:44.210977    2579 scope.go:117] "RemoveContainer" containerID="efc06c8335aa8d99af0771775c9201d7e712575a7a754abfc138fe1b3504f126"
Sep 09 12:41:44.262199 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:44.262161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:41:49.341233 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:41:49.341201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:13.676995 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:13.676897    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:26.044996 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:26.044954    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:27.010455 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:27.010413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:33.753546 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:33.753495    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:33.851161 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:33.851114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:34.029125 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:34.029035    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:35.209033 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:35.208996    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:35.312913 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:35.312873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:35.409593 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:35.409559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:36.020701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:36.020664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:36.124391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:36.124359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:36.220530 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:36.220492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:37.444959 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:37.444924    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:43.581834 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:43.581798    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:43.683650 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:43.683604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:44.234515 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:44.234480    2579 scope.go:117] "RemoveContainer" containerID="bbe13754d52405d712ed0ca00c529d02dbe3b4c2fab9587263a7d9958258f4d1"
Sep 09 12:42:44.242657 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:44.242626    2579 scope.go:117] "RemoveContainer" containerID="0c208e0af48d97258458f541c732b095636335e15be58b38e8bc8fbdcbcd7ff5"
Sep 09 12:42:44.276597 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:44.276559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:50.518081 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:50.518042    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:50.625212 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:50.625179    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:50.719153 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:50.719088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:52.464741 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:52.464708    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:42:53.470534 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:42:53.470496    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:01.205248 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:01.205211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:01.460075 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:01.459975    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:05.546207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:05.546176    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:22.429247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:22.429213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:29.303722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:29.303683    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:29.414145 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:29.414107    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:29.680496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:29.680381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:29.995696 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:29.995631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:30.550311 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:30.550275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:33.136356 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:33.136321    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:33.228970 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:33.228939    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:34.251330 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.251293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:34.350429 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.350392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:34.707391 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.707348    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:34.961139 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.961057    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"]
Sep 09 12:43:34.961478 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.961461    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4c4d2e-6833-4a1a-9b41-917559799793" containerName="revoke-keys"
Sep 09 12:43:34.961564 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.961481    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4c4d2e-6833-4a1a-9b41-917559799793" containerName="revoke-keys"
Sep 09 12:43:34.961564 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.961524    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aca109c5-e333-4462-aafc-30b8c7278b04" containerName="payload-pre-processing"
Sep 09 12:43:34.961564 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.961533    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aca109c5-e333-4462-aafc-30b8c7278b04" containerName="payload-pre-processing"
Sep 09 12:43:34.961745 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.961615    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aca109c5-e333-4462-aafc-30b8c7278b04" containerName="payload-pre-processing"
Sep 09 12:43:34.961745 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.961630    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4c4d2e-6833-4a1a-9b41-917559799793" containerName="revoke-keys"
Sep 09 12:43:34.964664 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.964630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:34.966868 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.966849    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ab8d88\""
Sep 09 12:43:34.966951 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.966855    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ab8d88-dockercfg-7sj5w\""
Sep 09 12:43:34.977205 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:34.977181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"]
Sep 09 12:43:35.065014 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.064972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbthx\" (UniqueName: \"kubernetes.io/projected/d56deb6b-1a50-449e-9885-c363bf99e39a-kube-api-access-wbthx\") pod \"payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps\" (UID: \"d56deb6b-1a50-449e-9885-c363bf99e39a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:35.065190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.065077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56deb6b-1a50-449e-9885-c363bf99e39a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps\" (UID: \"d56deb6b-1a50-449e-9885-c363bf99e39a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:35.127096 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.127064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:35.165923 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.165891    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbthx\" (UniqueName: \"kubernetes.io/projected/d56deb6b-1a50-449e-9885-c363bf99e39a-kube-api-access-wbthx\") pod \"payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps\" (UID: \"d56deb6b-1a50-449e-9885-c363bf99e39a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:35.166091 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.165961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56deb6b-1a50-449e-9885-c363bf99e39a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps\" (UID: \"d56deb6b-1a50-449e-9885-c363bf99e39a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:35.166705 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.166684    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56deb6b-1a50-449e-9885-c363bf99e39a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps\" (UID: \"d56deb6b-1a50-449e-9885-c363bf99e39a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:35.173958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.173940    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbthx\" (UniqueName: \"kubernetes.io/projected/d56deb6b-1a50-449e-9885-c363bf99e39a-kube-api-access-wbthx\") pod \"payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps\" (UID: \"d56deb6b-1a50-449e-9885-c363bf99e39a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:35.199710 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.199670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:35.274987 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.274916    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:35.544152 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:35.544120    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"]
Sep 09 12:43:35.545597 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:43:35.545564    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd56deb6b_1a50_449e_9885_c363bf99e39a.slice/crio-177b14a32e0eac3828039759b1e51a96b9a191ebe1a7485a2b3b83597afe49fc WatchSource:0}: Error finding container 177b14a32e0eac3828039759b1e51a96b9a191ebe1a7485a2b3b83597afe49fc: Status 404 returned error can't find the container with id 177b14a32e0eac3828039759b1e51a96b9a191ebe1a7485a2b3b83597afe49fc
Sep 09 12:43:36.457958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:36.457873    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps" event={"ID":"d56deb6b-1a50-449e-9885-c363bf99e39a","Type":"ContainerStarted","Data":"0dbbdfcd75f23c2c6a7488b63a04025366ccbf7476463f14dfbf293bf2434927"}
Sep 09 12:43:36.457958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:36.457931    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:36.457958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:36.457946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps" event={"ID":"d56deb6b-1a50-449e-9885-c363bf99e39a","Type":"ContainerStarted","Data":"177b14a32e0eac3828039759b1e51a96b9a191ebe1a7485a2b3b83597afe49fc"}
Sep 09 12:43:36.474963 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:36.474911    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps" podStartSLOduration=2.474893924 podStartE2EDuration="2.474893924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:43:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:43:36.474714552 +0000 UTC m=+1432.798935088" watchObservedRunningTime="2026-09-09 12:43:36.474893924 +0000 UTC m=+1432.799114456"
Sep 09 12:43:36.784272 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:36.784168    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:36.884147 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:36.884114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:47.466237 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:47.466194    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:43:58.071013 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.070979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:58.578175 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.578113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:58.876260 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.876162    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"]
Sep 09 12:43:58.879579 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.879562    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.881750 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.881723    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-3f4a7d-openshift-default-dockercfg-sd2q8\""
Sep 09 12:43:58.894383 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.894350    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"]
Sep 09 12:43:58.961367 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.961331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:43:58.970556 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88080e7-f4f7-4913-ab9b-85d173ca533f-istiod-ca-cert\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.970556 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970566    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lnvh\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-kube-api-access-2lnvh\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.970839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-credential-socket\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.970839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-envoy\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.970839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-token\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.970839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970767    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-socket\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.970839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-podinfo\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.971115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-data\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:58.971115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:58.970948    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-certs\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.071706 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.071630    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-certs\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.071706 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.071703    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88080e7-f4f7-4913-ab9b-85d173ca533f-istiod-ca-cert\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.071723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lnvh\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-kube-api-access-2lnvh\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.071741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-credential-socket\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.071762    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-envoy\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.071974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-token\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.072062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-socket\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.072088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-podinfo\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072256 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.072137    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-data\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.072419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-certs\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.072442    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-credential-socket\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.072504    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-data\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.072574    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-socket\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.072861 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.072800    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88080e7-f4f7-4913-ab9b-85d173ca533f-istiod-ca-cert\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.074057 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.074038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-envoy\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.074436 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.074418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-podinfo\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.082289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.082265    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-token\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.082414 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.082331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lnvh\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-kube-api-access-2lnvh\") pod \"e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") " pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.189856 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.189765    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:43:59.473904 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.473882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"]
Sep 09 12:43:59.476607 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:43:59.476584    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode88080e7_f4f7_4913_ab9b_85d173ca533f.slice/crio-2de9a9a2030d7df14e925926815b3f8d6511bdca59b2a8ac1e20a6b0ba910f2d WatchSource:0}: Error finding container 2de9a9a2030d7df14e925926815b3f8d6511bdca59b2a8ac1e20a6b0ba910f2d: Status 404 returned error can't find the container with id 2de9a9a2030d7df14e925926815b3f8d6511bdca59b2a8ac1e20a6b0ba910f2d
Sep 09 12:43:59.478968 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.478937    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:43:59.479035 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.479012    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:43:59.479072 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.479051    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:43:59.578444 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.578409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" event={"ID":"e88080e7-f4f7-4913-ab9b-85d173ca533f","Type":"ContainerStarted","Data":"c6e0c9a6865d8922d03e0cda6ffdd148bcf3b03973728f67e0b709e27566c800"}
Sep 09 12:43:59.578647 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.578473    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" event={"ID":"e88080e7-f4f7-4913-ab9b-85d173ca533f","Type":"ContainerStarted","Data":"2de9a9a2030d7df14e925926815b3f8d6511bdca59b2a8ac1e20a6b0ba910f2d"}
Sep 09 12:43:59.600989 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:43:59.600915    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" podStartSLOduration=1.600889273 podStartE2EDuration="1.600889273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:43:59.597909182 +0000 UTC m=+1455.922129714" watchObservedRunningTime="2026-09-09 12:43:59.600889273 +0000 UTC m=+1455.925109805"
Sep 09 12:44:00.062794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:00.062741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:00.190255 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:00.190218    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:44:00.195224 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:00.195195    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:44:00.584184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:00.584153    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:44:00.585230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:00.585212    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:44:01.541764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:01.541710    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:01.691527 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:01.691489    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:02.368400 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:02.368364    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:02.814572 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:02.814537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:02.940362 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:02.940328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:03.079599 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.079504    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:03.311770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.311734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:03.406819 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.406734    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"]
Sep 09 12:44:03.410089 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.410068    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:03.412156 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.412130    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-3f4a7d\""
Sep 09 12:44:03.412156 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.412130    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-3f4a7d-dockercfg-gbb9l\""
Sep 09 12:44:03.419506 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.419469    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:03.424050 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.424022    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"]
Sep 09 12:44:03.512769 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.512731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b771071e-0b38-41a5-801c-2e29dc40296a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777\" (UID: \"b771071e-0b38-41a5-801c-2e29dc40296a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:03.513492 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.513448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fflpc\" (UniqueName: \"kubernetes.io/projected/b771071e-0b38-41a5-801c-2e29dc40296a-kube-api-access-fflpc\") pod \"payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777\" (UID: \"b771071e-0b38-41a5-801c-2e29dc40296a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:03.614958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.614915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fflpc\" (UniqueName: \"kubernetes.io/projected/b771071e-0b38-41a5-801c-2e29dc40296a-kube-api-access-fflpc\") pod \"payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777\" (UID: \"b771071e-0b38-41a5-801c-2e29dc40296a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:03.615112 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.615058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b771071e-0b38-41a5-801c-2e29dc40296a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777\" (UID: \"b771071e-0b38-41a5-801c-2e29dc40296a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:03.615964 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.615941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b771071e-0b38-41a5-801c-2e29dc40296a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777\" (UID: \"b771071e-0b38-41a5-801c-2e29dc40296a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:03.623295 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.623271    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fflpc\" (UniqueName: \"kubernetes.io/projected/b771071e-0b38-41a5-801c-2e29dc40296a-kube-api-access-fflpc\") pod \"payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777\" (UID: \"b771071e-0b38-41a5-801c-2e29dc40296a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:03.720750 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:03.720709    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:04.002299 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:04.002267    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"]
Sep 09 12:44:04.003585 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:44:04.003551    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb771071e_0b38_41a5_801c_2e29dc40296a.slice/crio-5e427fec123f6f9a434945df04e7f84dd0d3f82e908c11f8feab8a1a1067f4b3 WatchSource:0}: Error finding container 5e427fec123f6f9a434945df04e7f84dd0d3f82e908c11f8feab8a1a1067f4b3: Status 404 returned error can't find the container with id 5e427fec123f6f9a434945df04e7f84dd0d3f82e908c11f8feab8a1a1067f4b3
Sep 09 12:44:04.605986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:04.605948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777" event={"ID":"b771071e-0b38-41a5-801c-2e29dc40296a","Type":"ContainerStarted","Data":"6ca8db4ea4233757943ccf97f3105f1883588a23cb0849e0ea66ed47cb2d53e7"}
Sep 09 12:44:04.606230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:04.606010    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:04.606230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:04.606031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777" event={"ID":"b771071e-0b38-41a5-801c-2e29dc40296a","Type":"ContainerStarted","Data":"5e427fec123f6f9a434945df04e7f84dd0d3f82e908c11f8feab8a1a1067f4b3"}
Sep 09 12:44:04.625527 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:04.625470    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777" podStartSLOduration=1.625451886 podStartE2EDuration="1.625451886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:44:04.622964659 +0000 UTC m=+1460.947185204" watchObservedRunningTime="2026-09-09 12:44:04.625451886 +0000 UTC m=+1460.949672419"
Sep 09 12:44:06.512771 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:06.512734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:15.613771 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:15.613728    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:44:21.703199 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:21.703163    2579 generic.go:356] "Generic (PLEG): container finished" podID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerID="c6e0c9a6865d8922d03e0cda6ffdd148bcf3b03973728f67e0b709e27566c800" exitCode=0
Sep 09 12:44:21.703619 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:21.703237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" event={"ID":"e88080e7-f4f7-4913-ab9b-85d173ca533f","Type":"ContainerDied","Data":"c6e0c9a6865d8922d03e0cda6ffdd148bcf3b03973728f67e0b709e27566c800"}
Sep 09 12:44:21.703696 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:21.703620    2579 scope.go:117] "RemoveContainer" containerID="c6e0c9a6865d8922d03e0cda6ffdd148bcf3b03973728f67e0b709e27566c800"
Sep 09 12:44:21.704819 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:21.704794    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:44:21.704886 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:21.704859    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:44:21.704924 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:21.704888    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:44:22.710550 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:22.710516    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" event={"ID":"e88080e7-f4f7-4913-ab9b-85d173ca533f","Type":"ContainerStarted","Data":"c2e006020b2734bbc3bb778ae5af08aff01c1ed106c368ae8827935e21c2a089"}
Sep 09 12:44:23.190879 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:23.190792    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:44:23.191056 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:23.191035    2579 patch_prober.go:28] interesting pod/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 12:44:23.191115 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:23.191097    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 12:44:24.190106 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:24.190066    2579 patch_prober.go:28] interesting pod/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 12:44:24.190513 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:24.190129    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 12:44:24.611548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:24.611507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:25.190648 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:25.190605    2579 patch_prober.go:28] interesting pod/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 12:44:25.191052 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:25.190707    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 12:44:25.400646 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:25.400610    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:26.190230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:26.190181    2579 patch_prober.go:28] interesting pod/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 12:44:26.190424 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:26.190247    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 12:44:26.606596 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:26.606554    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:27.191021 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:27.190981    2579 patch_prober.go:28] interesting pod/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 09 12:44:27.191211 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:27.191055    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 09 12:44:28.195687 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:28.195658    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:44:28.196124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:28.195947    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:44:28.196847 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:28.196828    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:44:29.512781 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.512742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:29.692491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.689765    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"]
Sep 09 12:44:29.694732 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.694696    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"
Sep 09 12:44:29.697209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.697182    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:44:29.697340 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.697309    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:44:29.702055 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.702017    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"]
Sep 09 12:44:29.743219 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.743170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/046333fa-2985-40cf-9440-cbe979b6535d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr\" (UID: \"046333fa-2985-40cf-9440-cbe979b6535d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"
Sep 09 12:44:29.844230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.844124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/046333fa-2985-40cf-9440-cbe979b6535d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr\" (UID: \"046333fa-2985-40cf-9440-cbe979b6535d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"
Sep 09 12:44:29.844778 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:29.844758    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/046333fa-2985-40cf-9440-cbe979b6535d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr\" (UID: \"046333fa-2985-40cf-9440-cbe979b6535d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"
Sep 09 12:44:30.009493 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:30.009451    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"
Sep 09 12:44:30.286918 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:30.286889    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"]
Sep 09 12:44:30.288819 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:44:30.288788    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod046333fa_2985_40cf_9440_cbe979b6535d.slice/crio-a7bc11d63cb2e25a42fe0ce11c5359e18104d1aaaa42c83236f7c6797add0e45 WatchSource:0}: Error finding container a7bc11d63cb2e25a42fe0ce11c5359e18104d1aaaa42c83236f7c6797add0e45: Status 404 returned error can't find the container with id a7bc11d63cb2e25a42fe0ce11c5359e18104d1aaaa42c83236f7c6797add0e45
Sep 09 12:44:30.518568 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:30.518528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:30.755877 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:30.755838    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr" event={"ID":"046333fa-2985-40cf-9440-cbe979b6535d","Type":"ContainerStarted","Data":"622b805ac7691066303d38521a9efe6ab4f522d2e983a4d7c60f1ad7da68048c"}
Sep 09 12:44:30.756053 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:30.755895    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr" event={"ID":"046333fa-2985-40cf-9440-cbe979b6535d","Type":"ContainerStarted","Data":"a7bc11d63cb2e25a42fe0ce11c5359e18104d1aaaa42c83236f7c6797add0e45"}
Sep 09 12:44:30.780381 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:30.780320    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr" podStartSLOduration=1.780304075 podStartE2EDuration="1.780304075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:44:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:44:30.778817071 +0000 UTC m=+1487.103037601" watchObservedRunningTime="2026-09-09 12:44:30.780304075 +0000 UTC m=+1487.104524606"
Sep 09 12:44:31.762220 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:31.762173    2579 generic.go:356] "Generic (PLEG): container finished" podID="046333fa-2985-40cf-9440-cbe979b6535d" containerID="622b805ac7691066303d38521a9efe6ab4f522d2e983a4d7c60f1ad7da68048c" exitCode=0
Sep 09 12:44:31.762668 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:31.762273    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr" event={"ID":"046333fa-2985-40cf-9440-cbe979b6535d","Type":"ContainerDied","Data":"622b805ac7691066303d38521a9efe6ab4f522d2e983a4d7c60f1ad7da68048c"}
Sep 09 12:44:32.905519 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:32.905483    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"
Sep 09 12:44:32.972712 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:32.972671    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/046333fa-2985-40cf-9440-cbe979b6535d-maas-api-service-ca\") pod \"046333fa-2985-40cf-9440-cbe979b6535d\" (UID: \"046333fa-2985-40cf-9440-cbe979b6535d\") "
Sep 09 12:44:32.973030 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:32.973007    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/046333fa-2985-40cf-9440-cbe979b6535d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "046333fa-2985-40cf-9440-cbe979b6535d" (UID: "046333fa-2985-40cf-9440-cbe979b6535d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:44:33.074221 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:33.074102    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/046333fa-2985-40cf-9440-cbe979b6535d-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:44:33.773871 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:33.773830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr" event={"ID":"046333fa-2985-40cf-9440-cbe979b6535d","Type":"ContainerDied","Data":"a7bc11d63cb2e25a42fe0ce11c5359e18104d1aaaa42c83236f7c6797add0e45"}
Sep 09 12:44:33.774063 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:33.773887    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7bc11d63cb2e25a42fe0ce11c5359e18104d1aaaa42c83236f7c6797add0e45"
Sep 09 12:44:33.774063 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:33.773844    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"
Sep 09 12:44:35.591203 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:35.591163    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:36.086701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:36.086669    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:36.560855 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:36.560819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:39.967597 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:39.967557    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:44.271977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:44.271946    2579 scope.go:117] "RemoveContainer" containerID="3f686ac5d7accc49da5e2f1c88a21557a11473e5664bb5539ef37ad3ff5a72f2"
Sep 09 12:44:44.282071 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:44.282049    2579 scope.go:117] "RemoveContainer" containerID="6109eab8f53dd4657a424fbaf61345dfa106c107076508fbc0b6847ae95e171c"
Sep 09 12:44:44.289750 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:44.289729    2579 scope.go:117] "RemoveContainer" containerID="cc8bebff0060f0f917fdc910e1c6d2da67da7529a005900b24da034db8b3cf67"
Sep 09 12:44:44.297418 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:44.297394    2579 scope.go:117] "RemoveContainer" containerID="7eaef7eb24aaf45435cb739f27e537f2d209ec458eb98cf38fd3888d7801f4e3"
Sep 09 12:44:45.568448 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.568414    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:45.626486 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.626457    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"]
Sep 09 12:44:45.626748 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.626726    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps" podUID="d56deb6b-1a50-449e-9885-c363bf99e39a" containerName="payload-pre-processing" containerID="cri-o://0dbbdfcd75f23c2c6a7488b63a04025366ccbf7476463f14dfbf293bf2434927" gracePeriod=30
Sep 09 12:44:45.634615 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.634257    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:45.696352 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.696317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:45.800285 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.800241    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:45.803988 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.803946    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"]
Sep 09 12:44:45.806153 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.806129    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ab8d88-9q4wr"]
Sep 09 12:44:45.838148 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.838079    2579 generic.go:356] "Generic (PLEG): container finished" podID="d56deb6b-1a50-449e-9885-c363bf99e39a" containerID="0dbbdfcd75f23c2c6a7488b63a04025366ccbf7476463f14dfbf293bf2434927" exitCode=0
Sep 09 12:44:45.838148 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.838114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps" event={"ID":"d56deb6b-1a50-449e-9885-c363bf99e39a","Type":"ContainerDied","Data":"0dbbdfcd75f23c2c6a7488b63a04025366ccbf7476463f14dfbf293bf2434927"}
Sep 09 12:44:45.890809 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.890778    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:44:45.903525 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.903474    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:45.998557 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.998523    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56deb6b-1a50-449e-9885-c363bf99e39a-plugins-config-volume\") pod \"d56deb6b-1a50-449e-9885-c363bf99e39a\" (UID: \"d56deb6b-1a50-449e-9885-c363bf99e39a\") "
Sep 09 12:44:45.998770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.998579    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbthx\" (UniqueName: \"kubernetes.io/projected/d56deb6b-1a50-449e-9885-c363bf99e39a-kube-api-access-wbthx\") pod \"d56deb6b-1a50-449e-9885-c363bf99e39a\" (UID: \"d56deb6b-1a50-449e-9885-c363bf99e39a\") "
Sep 09 12:44:45.999025 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:45.998997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d56deb6b-1a50-449e-9885-c363bf99e39a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d56deb6b-1a50-449e-9885-c363bf99e39a" (UID: "d56deb6b-1a50-449e-9885-c363bf99e39a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:44:46.000613 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.000592    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d56deb6b-1a50-449e-9885-c363bf99e39a-kube-api-access-wbthx" (OuterVolumeSpecName: "kube-api-access-wbthx") pod "d56deb6b-1a50-449e-9885-c363bf99e39a" (UID: "d56deb6b-1a50-449e-9885-c363bf99e39a"). InnerVolumeSpecName "kube-api-access-wbthx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:44:46.099449 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.099369    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56deb6b-1a50-449e-9885-c363bf99e39a-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:44:46.099449 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.099399    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbthx\" (UniqueName: \"kubernetes.io/projected/d56deb6b-1a50-449e-9885-c363bf99e39a-kube-api-access-wbthx\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:44:46.104003 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.103974    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="046333fa-2985-40cf-9440-cbe979b6535d" path="/var/lib/kubelet/pods/046333fa-2985-40cf-9440-cbe979b6535d/volumes"
Sep 09 12:44:46.181669 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.181620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:46.843748 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.843720    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"
Sep 09 12:44:46.844296 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.843765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps" event={"ID":"d56deb6b-1a50-449e-9885-c363bf99e39a","Type":"ContainerDied","Data":"177b14a32e0eac3828039759b1e51a96b9a191ebe1a7485a2b3b83597afe49fc"}
Sep 09 12:44:46.844296 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.843837    2579 scope.go:117] "RemoveContainer" containerID="0dbbdfcd75f23c2c6a7488b63a04025366ccbf7476463f14dfbf293bf2434927"
Sep 09 12:44:46.860580 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.860551    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"]
Sep 09 12:44:46.866598 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:46.866572    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ab8d88-57b6f45f98-5jsps"]
Sep 09 12:44:48.102538 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:48.102500    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d56deb6b-1a50-449e-9885-c363bf99e39a" path="/var/lib/kubelet/pods/d56deb6b-1a50-449e-9885-c363bf99e39a/volumes"
Sep 09 12:44:52.525830 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:52.525795    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:52.626503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:52.626465    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:52.790548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:52.790457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:53.014096 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.014063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:53.149861 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.149770    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"]
Sep 09 12:44:53.150148 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.150135    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d56deb6b-1a50-449e-9885-c363bf99e39a" containerName="payload-pre-processing"
Sep 09 12:44:53.150191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.150149    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d56deb6b-1a50-449e-9885-c363bf99e39a" containerName="payload-pre-processing"
Sep 09 12:44:53.150191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.150165    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="046333fa-2985-40cf-9440-cbe979b6535d" containerName="revoke-keys"
Sep 09 12:44:53.150191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.150171    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="046333fa-2985-40cf-9440-cbe979b6535d" containerName="revoke-keys"
Sep 09 12:44:53.150305 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.150239    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d56deb6b-1a50-449e-9885-c363bf99e39a" containerName="payload-pre-processing"
Sep 09 12:44:53.150305 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.150251    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="046333fa-2985-40cf-9440-cbe979b6535d" containerName="revoke-keys"
Sep 09 12:44:53.154548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.154523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"
Sep 09 12:44:53.156921 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.156890    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:44:53.157076 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.156985    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:44:53.160548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.160520    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"]
Sep 09 12:44:53.254797 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.254759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:53.265938 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.265900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5a382c-6e3f-45eb-842f-805098b20212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2\" (UID: \"ed5a382c-6e3f-45eb-842f-805098b20212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"
Sep 09 12:44:53.367524 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.367480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5a382c-6e3f-45eb-842f-805098b20212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2\" (UID: \"ed5a382c-6e3f-45eb-842f-805098b20212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"
Sep 09 12:44:53.368170 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.368146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5a382c-6e3f-45eb-842f-805098b20212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2\" (UID: \"ed5a382c-6e3f-45eb-842f-805098b20212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"
Sep 09 12:44:53.466471 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.466435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"
Sep 09 12:44:53.749041 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.748996    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"]
Sep 09 12:44:53.751342 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:44:53.751306    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5a382c_6e3f_45eb_842f_805098b20212.slice/crio-e196f86ae9f4f23c1ca8acfc2fc7e7646ef70a5df91ad59895644943b0bad9b1 WatchSource:0}: Error finding container e196f86ae9f4f23c1ca8acfc2fc7e7646ef70a5df91ad59895644943b0bad9b1: Status 404 returned error can't find the container with id e196f86ae9f4f23c1ca8acfc2fc7e7646ef70a5df91ad59895644943b0bad9b1
Sep 09 12:44:53.883078 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:53.883026    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2" event={"ID":"ed5a382c-6e3f-45eb-842f-805098b20212","Type":"ContainerStarted","Data":"e196f86ae9f4f23c1ca8acfc2fc7e7646ef70a5df91ad59895644943b0bad9b1"}
Sep 09 12:44:54.632512 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:54.632474    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:44:54.889890 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:54.889797    2579 generic.go:356] "Generic (PLEG): container finished" podID="ed5a382c-6e3f-45eb-842f-805098b20212" containerID="969b22b31d8c8a2425e4055d4eca5954731d107a370eb4c3913a1f089ee8ad4d" exitCode=0
Sep 09 12:44:54.889890 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:54.889861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2" event={"ID":"ed5a382c-6e3f-45eb-842f-805098b20212","Type":"ContainerDied","Data":"969b22b31d8c8a2425e4055d4eca5954731d107a370eb4c3913a1f089ee8ad4d"}
Sep 09 12:44:56.026004 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:56.025974    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"
Sep 09 12:44:56.193159 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:56.193075    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5a382c-6e3f-45eb-842f-805098b20212-maas-api-service-ca\") pod \"ed5a382c-6e3f-45eb-842f-805098b20212\" (UID: \"ed5a382c-6e3f-45eb-842f-805098b20212\") "
Sep 09 12:44:56.193426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:56.193401    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed5a382c-6e3f-45eb-842f-805098b20212-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed5a382c-6e3f-45eb-842f-805098b20212" (UID: "ed5a382c-6e3f-45eb-842f-805098b20212"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:44:56.294690 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:56.294618    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed5a382c-6e3f-45eb-842f-805098b20212-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:44:56.900433 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:56.900402    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"
Sep 09 12:44:56.900611 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:56.900435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2" event={"ID":"ed5a382c-6e3f-45eb-842f-805098b20212","Type":"ContainerDied","Data":"e196f86ae9f4f23c1ca8acfc2fc7e7646ef70a5df91ad59895644943b0bad9b1"}
Sep 09 12:44:56.900611 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:44:56.900482    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e196f86ae9f4f23c1ca8acfc2fc7e7646ef70a5df91ad59895644943b0bad9b1"
Sep 09 12:45:00.154699 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.154654    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"]
Sep 09 12:45:00.155140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.155065    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed5a382c-6e3f-45eb-842f-805098b20212" containerName="revoke-keys"
Sep 09 12:45:00.155140 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.155078    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed5a382c-6e3f-45eb-842f-805098b20212" containerName="revoke-keys"
Sep 09 12:45:00.155219 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.155185    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed5a382c-6e3f-45eb-842f-805098b20212" containerName="revoke-keys"
Sep 09 12:45:00.158259 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.158237    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"]
Sep 09 12:45:00.158383 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.158362    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"
Sep 09 12:45:00.160672 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.160652    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:45:00.161454 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.161440    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl"]
Sep 09 12:45:00.161584 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.161565    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"
Sep 09 12:45:00.164724 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.164705    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl"
Sep 09 12:45:00.168576 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.168548    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"]
Sep 09 12:45:00.170894 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.170873    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"
Sep 09 12:45:00.174409 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.174152    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl"]
Sep 09 12:45:00.176659 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.176614    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"]
Sep 09 12:45:00.183949 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.183926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"
Sep 09 12:45:00.228934 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.228876    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl"
Sep 09 12:45:00.461107 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.461060    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"]
Sep 09 12:45:00.464078 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:45:00.464051    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb78d50_753f_483e_8f43_6ef6f6d78cf9.slice/crio-d44c9903e9b633d67e1f14d5db24911aa786e4fab4a4831b82f59853a51d2d5e WatchSource:0}: Error finding container d44c9903e9b633d67e1f14d5db24911aa786e4fab4a4831b82f59853a51d2d5e: Status 404 returned error can't find the container with id d44c9903e9b633d67e1f14d5db24911aa786e4fab4a4831b82f59853a51d2d5e
Sep 09 12:45:00.474460 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.474438    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"]
Sep 09 12:45:00.477062 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:45:00.477033    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7a1e8b_d631_4c1f_896a_adad71bef3fa.slice/crio-677a59afec1841ecad87548c468ca92b567455704ca00c22c64ffaaef134517c WatchSource:0}: Error finding container 677a59afec1841ecad87548c468ca92b567455704ca00c22c64ffaaef134517c: Status 404 returned error can't find the container with id 677a59afec1841ecad87548c468ca92b567455704ca00c22c64ffaaef134517c
Sep 09 12:45:00.527225 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.527200    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl"]
Sep 09 12:45:00.529085 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:45:00.529056    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a9e3de_3508_47db_bd8e_58691ce92c7c.slice/crio-8ee7c3724eb3f1c7f639c0c41ae7724ea9e06348afa72e299a7f58a89731d71c WatchSource:0}: Error finding container 8ee7c3724eb3f1c7f639c0c41ae7724ea9e06348afa72e299a7f58a89731d71c: Status 404 returned error can't find the container with id 8ee7c3724eb3f1c7f639c0c41ae7724ea9e06348afa72e299a7f58a89731d71c
Sep 09 12:45:00.922765 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.922723    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl" event={"ID":"60a9e3de-3508-47db-bd8e-58691ce92c7c","Type":"ContainerStarted","Data":"38692255a29c6c41bcabaf5b90ee32e973128c2feb21d22e90c8a49f0bf35f0b"}
Sep 09 12:45:00.922962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.922782    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl" event={"ID":"60a9e3de-3508-47db-bd8e-58691ce92c7c","Type":"ContainerStarted","Data":"8ee7c3724eb3f1c7f639c0c41ae7724ea9e06348afa72e299a7f58a89731d71c"}
Sep 09 12:45:00.924348 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.924323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t" event={"ID":"7fb78d50-753f-483e-8f43-6ef6f6d78cf9","Type":"ContainerStarted","Data":"cc9502990f06222e2085195268619e7a18a0fae0423a43be61d0e14110c03f00"}
Sep 09 12:45:00.924485 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.924367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t" event={"ID":"7fb78d50-753f-483e-8f43-6ef6f6d78cf9","Type":"ContainerStarted","Data":"d44c9903e9b633d67e1f14d5db24911aa786e4fab4a4831b82f59853a51d2d5e"}
Sep 09 12:45:00.925862 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.925839    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v" event={"ID":"7e7a1e8b-d631-4c1f-896a-adad71bef3fa","Type":"ContainerStarted","Data":"b37955e494472a6d97c5ca288c49af402f19da0bcf1f0c646fcacd89435d0a53"}
Sep 09 12:45:00.925930 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.925884    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v" event={"ID":"7e7a1e8b-d631-4c1f-896a-adad71bef3fa","Type":"ContainerStarted","Data":"677a59afec1841ecad87548c468ca92b567455704ca00c22c64ffaaef134517c"}
Sep 09 12:45:00.940760 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.940706    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl" podStartSLOduration=0.940688534 podStartE2EDuration="940.688534ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:45:00.939552155 +0000 UTC m=+1517.263772686" watchObservedRunningTime="2026-09-09 12:45:00.940688534 +0000 UTC m=+1517.264909067"
Sep 09 12:45:00.956824 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.956769    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v" podStartSLOduration=0.956751455 podStartE2EDuration="956.751455ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:45:00.955198043 +0000 UTC m=+1517.279418575" watchObservedRunningTime="2026-09-09 12:45:00.956751455 +0000 UTC m=+1517.280972032"
Sep 09 12:45:00.972531 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:00.972479    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t" podStartSLOduration=0.972461034 podStartE2EDuration="972.461034ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:45:00.970700963 +0000 UTC m=+1517.294921497" watchObservedRunningTime="2026-09-09 12:45:00.972461034 +0000 UTC m=+1517.296681565"
Sep 09 12:45:01.940217 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:01.940111    2579 generic.go:356] "Generic (PLEG): container finished" podID="7fb78d50-753f-483e-8f43-6ef6f6d78cf9" containerID="cc9502990f06222e2085195268619e7a18a0fae0423a43be61d0e14110c03f00" exitCode=0
Sep 09 12:45:01.940630 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:01.940238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t" event={"ID":"7fb78d50-753f-483e-8f43-6ef6f6d78cf9","Type":"ContainerDied","Data":"cc9502990f06222e2085195268619e7a18a0fae0423a43be61d0e14110c03f00"}
Sep 09 12:45:01.942035 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:01.942009    2579 generic.go:356] "Generic (PLEG): container finished" podID="7e7a1e8b-d631-4c1f-896a-adad71bef3fa" containerID="b37955e494472a6d97c5ca288c49af402f19da0bcf1f0c646fcacd89435d0a53" exitCode=0
Sep 09 12:45:01.942149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:01.942080    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v" event={"ID":"7e7a1e8b-d631-4c1f-896a-adad71bef3fa","Type":"ContainerDied","Data":"b37955e494472a6d97c5ca288c49af402f19da0bcf1f0c646fcacd89435d0a53"}
Sep 09 12:45:01.943765 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:01.943741    2579 generic.go:356] "Generic (PLEG): container finished" podID="60a9e3de-3508-47db-bd8e-58691ce92c7c" containerID="38692255a29c6c41bcabaf5b90ee32e973128c2feb21d22e90c8a49f0bf35f0b" exitCode=0
Sep 09 12:45:01.943867 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:01.943788    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl" event={"ID":"60a9e3de-3508-47db-bd8e-58691ce92c7c","Type":"ContainerDied","Data":"38692255a29c6c41bcabaf5b90ee32e973128c2feb21d22e90c8a49f0bf35f0b"}
Sep 09 12:45:03.082593 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.082567    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"
Sep 09 12:45:03.132955 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.132930    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl"
Sep 09 12:45:03.136044 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.136027    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"
Sep 09 12:45:03.956499 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.956470    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"
Sep 09 12:45:03.956713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.956516    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v" event={"ID":"7e7a1e8b-d631-4c1f-896a-adad71bef3fa","Type":"ContainerDied","Data":"677a59afec1841ecad87548c468ca92b567455704ca00c22c64ffaaef134517c"}
Sep 09 12:45:03.956713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.956569    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="677a59afec1841ecad87548c468ca92b567455704ca00c22c64ffaaef134517c"
Sep 09 12:45:03.957810 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.957788    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl"
Sep 09 12:45:03.957927 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.957794    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815965-5khpl" event={"ID":"60a9e3de-3508-47db-bd8e-58691ce92c7c","Type":"ContainerDied","Data":"8ee7c3724eb3f1c7f639c0c41ae7724ea9e06348afa72e299a7f58a89731d71c"}
Sep 09 12:45:03.957927 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.957899    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ee7c3724eb3f1c7f639c0c41ae7724ea9e06348afa72e299a7f58a89731d71c"
Sep 09 12:45:03.958963 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.958941    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t" event={"ID":"7fb78d50-753f-483e-8f43-6ef6f6d78cf9","Type":"ContainerDied","Data":"d44c9903e9b633d67e1f14d5db24911aa786e4fab4a4831b82f59853a51d2d5e"}
Sep 09 12:45:03.959098 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.958977    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"
Sep 09 12:45:03.959098 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:03.958984    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d44c9903e9b633d67e1f14d5db24911aa786e4fab4a4831b82f59853a51d2d5e"
Sep 09 12:45:08.765191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.764993    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:08.778654 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.778610    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"]
Sep 09 12:45:08.778924 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.778898    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777" podUID="b771071e-0b38-41a5-801c-2e29dc40296a" containerName="payload-pre-processing" containerID="cri-o://6ca8db4ea4233757943ccf97f3105f1883588a23cb0849e0ea66ed47cb2d53e7" gracePeriod=30
Sep 09 12:45:08.872148 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.872116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:08.954757 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.954704    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"]
Sep 09 12:45:08.958183 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.958155    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-3f4a7d-pqvn2"]
Sep 09 12:45:08.958781 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.958699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:08.985100 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.985063    2579 generic.go:356] "Generic (PLEG): container finished" podID="b771071e-0b38-41a5-801c-2e29dc40296a" containerID="6ca8db4ea4233757943ccf97f3105f1883588a23cb0849e0ea66ed47cb2d53e7" exitCode=0
Sep 09 12:45:08.985245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:08.985135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777" event={"ID":"b771071e-0b38-41a5-801c-2e29dc40296a","Type":"ContainerDied","Data":"6ca8db4ea4233757943ccf97f3105f1883588a23cb0849e0ea66ed47cb2d53e7"}
Sep 09 12:45:09.031891 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.031869    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:45:09.036887 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.036849    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:09.104960 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.104925    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fflpc\" (UniqueName: \"kubernetes.io/projected/b771071e-0b38-41a5-801c-2e29dc40296a-kube-api-access-fflpc\") pod \"b771071e-0b38-41a5-801c-2e29dc40296a\" (UID: \"b771071e-0b38-41a5-801c-2e29dc40296a\") "
Sep 09 12:45:09.107681 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.107623    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b771071e-0b38-41a5-801c-2e29dc40296a-kube-api-access-fflpc" (OuterVolumeSpecName: "kube-api-access-fflpc") pod "b771071e-0b38-41a5-801c-2e29dc40296a" (UID: "b771071e-0b38-41a5-801c-2e29dc40296a"). InnerVolumeSpecName "kube-api-access-fflpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:09.205467 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.205437    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b771071e-0b38-41a5-801c-2e29dc40296a-plugins-config-volume\") pod \"b771071e-0b38-41a5-801c-2e29dc40296a\" (UID: \"b771071e-0b38-41a5-801c-2e29dc40296a\") "
Sep 09 12:45:09.205620 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.205561    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fflpc\" (UniqueName: \"kubernetes.io/projected/b771071e-0b38-41a5-801c-2e29dc40296a-kube-api-access-fflpc\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:09.205798 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.205775    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b771071e-0b38-41a5-801c-2e29dc40296a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b771071e-0b38-41a5-801c-2e29dc40296a" (UID: "b771071e-0b38-41a5-801c-2e29dc40296a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:09.306423 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.306342    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b771071e-0b38-41a5-801c-2e29dc40296a-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:09.339217 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.339186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:09.991568 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.991535    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"
Sep 09 12:45:09.991568 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.991558    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777" event={"ID":"b771071e-0b38-41a5-801c-2e29dc40296a","Type":"ContainerDied","Data":"5e427fec123f6f9a434945df04e7f84dd0d3f82e908c11f8feab8a1a1067f4b3"}
Sep 09 12:45:09.992069 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:09.991614    2579 scope.go:117] "RemoveContainer" containerID="6ca8db4ea4233757943ccf97f3105f1883588a23cb0849e0ea66ed47cb2d53e7"
Sep 09 12:45:10.018138 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:10.018097    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"]
Sep 09 12:45:10.020378 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:10.020358    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-3f4a7d-67c959496b-kv777"]
Sep 09 12:45:10.103062 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:10.103026    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b771071e-0b38-41a5-801c-2e29dc40296a" path="/var/lib/kubelet/pods/b771071e-0b38-41a5-801c-2e29dc40296a/volumes"
Sep 09 12:45:10.103409 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:10.103394    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed5a382c-6e3f-45eb-842f-805098b20212" path="/var/lib/kubelet/pods/ed5a382c-6e3f-45eb-842f-805098b20212/volumes"
Sep 09 12:45:15.628587 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:15.628552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:15.706114 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:15.706073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:15.876056 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:15.876021    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"]
Sep 09 12:45:15.876291 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:15.876265    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy" containerID="cri-o://c2e006020b2734bbc3bb778ae5af08aff01c1ed106c368ae8827935e21c2a089" gracePeriod=30
Sep 09 12:45:15.905674 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:15.905576    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:16.239927 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.239893    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"]
Sep 09 12:45:16.240242 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240230    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b771071e-0b38-41a5-801c-2e29dc40296a" containerName="payload-pre-processing"
Sep 09 12:45:16.240294 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240243    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b771071e-0b38-41a5-801c-2e29dc40296a" containerName="payload-pre-processing"
Sep 09 12:45:16.240294 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240262    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fb78d50-753f-483e-8f43-6ef6f6d78cf9" containerName="cleanup"
Sep 09 12:45:16.240294 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240267    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fb78d50-753f-483e-8f43-6ef6f6d78cf9" containerName="cleanup"
Sep 09 12:45:16.240294 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240281    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60a9e3de-3508-47db-bd8e-58691ce92c7c" containerName="cleanup"
Sep 09 12:45:16.240294 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240287    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="60a9e3de-3508-47db-bd8e-58691ce92c7c" containerName="cleanup"
Sep 09 12:45:16.240294 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240295    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e7a1e8b-d631-4c1f-896a-adad71bef3fa" containerName="cleanup"
Sep 09 12:45:16.240496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240301    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e7a1e8b-d631-4c1f-896a-adad71bef3fa" containerName="cleanup"
Sep 09 12:45:16.240496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240350    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fb78d50-753f-483e-8f43-6ef6f6d78cf9" containerName="cleanup"
Sep 09 12:45:16.240496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240359    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="60a9e3de-3508-47db-bd8e-58691ce92c7c" containerName="cleanup"
Sep 09 12:45:16.240496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240366    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b771071e-0b38-41a5-801c-2e29dc40296a" containerName="payload-pre-processing"
Sep 09 12:45:16.240496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.240373    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e7a1e8b-d631-4c1f-896a-adad71bef3fa" containerName="cleanup"
Sep 09 12:45:16.247897 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.247864    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"
Sep 09 12:45:16.250240 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.250185    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:45:16.250387 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.250185    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:45:16.252995 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.252963    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"]
Sep 09 12:45:16.258744 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.258720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7801ff3-9798-4968-9fff-205f5f6c70e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn\" (UID: \"b7801ff3-9798-4968-9fff-205f5f6c70e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"
Sep 09 12:45:16.359734 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.359693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7801ff3-9798-4968-9fff-205f5f6c70e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn\" (UID: \"b7801ff3-9798-4968-9fff-205f5f6c70e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"
Sep 09 12:45:16.360291 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.360273    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7801ff3-9798-4968-9fff-205f5f6c70e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn\" (UID: \"b7801ff3-9798-4968-9fff-205f5f6c70e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"
Sep 09 12:45:16.471344 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.471307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:16.559456 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.559361    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"
Sep 09 12:45:16.835777 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:16.835695    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"]
Sep 09 12:45:16.837978 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:45:16.837951    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7801ff3_9798_4968_9fff_205f5f6c70e0.slice/crio-4979e295e16cd44e2410de03ced4a22e132cd6e3e5b52df513367c361cc91fd3 WatchSource:0}: Error finding container 4979e295e16cd44e2410de03ced4a22e132cd6e3e5b52df513367c361cc91fd3: Status 404 returned error can't find the container with id 4979e295e16cd44e2410de03ced4a22e132cd6e3e5b52df513367c361cc91fd3
Sep 09 12:45:17.029003 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:17.028970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn" event={"ID":"b7801ff3-9798-4968-9fff-205f5f6c70e0","Type":"ContainerStarted","Data":"4979e295e16cd44e2410de03ced4a22e132cd6e3e5b52df513367c361cc91fd3"}
Sep 09 12:45:18.035515 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:18.035476    2579 generic.go:356] "Generic (PLEG): container finished" podID="b7801ff3-9798-4968-9fff-205f5f6c70e0" containerID="866044ff50db3893bd74d002a9784817557b33383ec0e1109f5cd85ed8aa88ff" exitCode=0
Sep 09 12:45:18.035911 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:18.035552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn" event={"ID":"b7801ff3-9798-4968-9fff-205f5f6c70e0","Type":"ContainerDied","Data":"866044ff50db3893bd74d002a9784817557b33383ec0e1109f5cd85ed8aa88ff"}
Sep 09 12:45:19.165459 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:19.165436    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"
Sep 09 12:45:19.186291 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:19.186265    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7801ff3-9798-4968-9fff-205f5f6c70e0-maas-api-service-ca\") pod \"b7801ff3-9798-4968-9fff-205f5f6c70e0\" (UID: \"b7801ff3-9798-4968-9fff-205f5f6c70e0\") "
Sep 09 12:45:19.186669 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:19.186630    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7801ff3-9798-4968-9fff-205f5f6c70e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7801ff3-9798-4968-9fff-205f5f6c70e0" (UID: "b7801ff3-9798-4968-9fff-205f5f6c70e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:19.287733 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:19.287694    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7801ff3-9798-4968-9fff-205f5f6c70e0-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:20.047034 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:20.047008    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"
Sep 09 12:45:20.047034 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:20.047027    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn" event={"ID":"b7801ff3-9798-4968-9fff-205f5f6c70e0","Type":"ContainerDied","Data":"4979e295e16cd44e2410de03ced4a22e132cd6e3e5b52df513367c361cc91fd3"}
Sep 09 12:45:20.047243 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:20.047071    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4979e295e16cd44e2410de03ced4a22e132cd6e3e5b52df513367c361cc91fd3"
Sep 09 12:45:21.055627 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.055597    2579 generic.go:356] "Generic (PLEG): container finished" podID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerID="c2e006020b2734bbc3bb778ae5af08aff01c1ed106c368ae8827935e21c2a089" exitCode=0
Sep 09 12:45:21.056062 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.055681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" event={"ID":"e88080e7-f4f7-4913-ab9b-85d173ca533f","Type":"ContainerDied","Data":"c2e006020b2734bbc3bb778ae5af08aff01c1ed106c368ae8827935e21c2a089"}
Sep 09 12:45:21.056062 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.055797    2579 scope.go:117] "RemoveContainer" containerID="c6e0c9a6865d8922d03e0cda6ffdd148bcf3b03973728f67e0b709e27566c800"
Sep 09 12:45:21.123581 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.123555    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:45:21.204377 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204352    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-token\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.204533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204398    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-certs\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.204533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204436    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-socket\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.204533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204490    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-data\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.204802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204533    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88080e7-f4f7-4913-ab9b-85d173ca533f-istiod-ca-cert\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.204802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204589    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lnvh\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-kube-api-access-2lnvh\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.204802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204625    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-envoy\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.204802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204672    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-podinfo\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.204802 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204706    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-credential-socket\") pod \"e88080e7-f4f7-4913-ab9b-85d173ca533f\" (UID: \"e88080e7-f4f7-4913-ab9b-85d173ca533f\") "
Sep 09 12:45:21.205048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204817    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:21.205048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204877    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:21.205048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204803    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:21.205224 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.205055    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:21.205413 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.205360    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-certs\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:21.205413 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.205387    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-workload-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:21.205413 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.205401    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-data\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:21.205920 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.204970    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e88080e7-f4f7-4913-ab9b-85d173ca533f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:21.206784 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.206756    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:21.206784 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.206776    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:21.207067 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.207048    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-kube-api-access-2lnvh" (OuterVolumeSpecName: "kube-api-access-2lnvh") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "kube-api-access-2lnvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:21.207120 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.207071    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e88080e7-f4f7-4913-ab9b-85d173ca533f" (UID: "e88080e7-f4f7-4913-ab9b-85d173ca533f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:45:21.306769 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.306733    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-envoy\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:21.306769 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.306762    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-podinfo\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:21.306769 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.306775    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e88080e7-f4f7-4913-ab9b-85d173ca533f-credential-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:21.306999 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.306787    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-istio-token\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:21.306999 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.306799    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e88080e7-f4f7-4913-ab9b-85d173ca533f-istiod-ca-cert\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:21.306999 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:21.306810    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lnvh\" (UniqueName: \"kubernetes.io/projected/e88080e7-f4f7-4913-ab9b-85d173ca533f-kube-api-access-2lnvh\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:22.062239 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:22.062206    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"
Sep 09 12:45:22.062723 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:22.062243    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq" event={"ID":"e88080e7-f4f7-4913-ab9b-85d173ca533f","Type":"ContainerDied","Data":"2de9a9a2030d7df14e925926815b3f8d6511bdca59b2a8ac1e20a6b0ba910f2d"}
Sep 09 12:45:22.062723 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:22.062306    2579 scope.go:117] "RemoveContainer" containerID="c2e006020b2734bbc3bb778ae5af08aff01c1ed106c368ae8827935e21c2a089"
Sep 09 12:45:22.085904 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:22.085880    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"]
Sep 09 12:45:22.090624 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:22.090599    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-3f4a7d-openshift-default-6475bfd948-t9dhq"]
Sep 09 12:45:22.101923 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:22.101896    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" path="/var/lib/kubelet/pods/e88080e7-f4f7-4913-ab9b-85d173ca533f/volumes"
Sep 09 12:45:22.264103 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:22.264072    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:26.881281 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:26.881245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:26.919142 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:26.919109    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"]
Sep 09 12:45:26.919386 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:26.919362    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2" podUID="bc4360ca-8972-4224-aedf-a7ee36202f36" containerName="payload-processing" containerID="cri-o://aaed71e59243c45d3525319172da47eb75fd1a88cfba6f71b205156c5233dcea" gracePeriod=30
Sep 09 12:45:27.089698 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.089651    2579 generic.go:356] "Generic (PLEG): container finished" podID="bc4360ca-8972-4224-aedf-a7ee36202f36" containerID="aaed71e59243c45d3525319172da47eb75fd1a88cfba6f71b205156c5233dcea" exitCode=0
Sep 09 12:45:27.089816 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.089729    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2" event={"ID":"bc4360ca-8972-4224-aedf-a7ee36202f36","Type":"ContainerDied","Data":"aaed71e59243c45d3525319172da47eb75fd1a88cfba6f71b205156c5233dcea"}
Sep 09 12:45:27.118657 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.118596    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"]
Sep 09 12:45:27.121415 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.121387    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-a3f43f-7tdmn"]
Sep 09 12:45:27.163957 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.163928    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:45:27.257454 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.257420    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4360ca-8972-4224-aedf-a7ee36202f36-plugins-config-volume\") pod \"bc4360ca-8972-4224-aedf-a7ee36202f36\" (UID: \"bc4360ca-8972-4224-aedf-a7ee36202f36\") "
Sep 09 12:45:27.257454 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.257457    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk2nr\" (UniqueName: \"kubernetes.io/projected/bc4360ca-8972-4224-aedf-a7ee36202f36-kube-api-access-sk2nr\") pod \"bc4360ca-8972-4224-aedf-a7ee36202f36\" (UID: \"bc4360ca-8972-4224-aedf-a7ee36202f36\") "
Sep 09 12:45:27.257914 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.257883    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc4360ca-8972-4224-aedf-a7ee36202f36-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc4360ca-8972-4224-aedf-a7ee36202f36" (UID: "bc4360ca-8972-4224-aedf-a7ee36202f36"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:27.260701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.260668    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc4360ca-8972-4224-aedf-a7ee36202f36-kube-api-access-sk2nr" (OuterVolumeSpecName: "kube-api-access-sk2nr") pod "bc4360ca-8972-4224-aedf-a7ee36202f36" (UID: "bc4360ca-8972-4224-aedf-a7ee36202f36"). InnerVolumeSpecName "kube-api-access-sk2nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:27.359173 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.359140    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc4360ca-8972-4224-aedf-a7ee36202f36-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:27.359173 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.359167    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk2nr\" (UniqueName: \"kubernetes.io/projected/bc4360ca-8972-4224-aedf-a7ee36202f36-kube-api-access-sk2nr\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:27.940837 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:27.940799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:28.040431 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:28.040395    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:28.095567 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:28.095539    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"
Sep 09 12:45:28.095742 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:28.095540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2" event={"ID":"bc4360ca-8972-4224-aedf-a7ee36202f36","Type":"ContainerDied","Data":"3dfd4cf2e1fe092e46584f1075b248c542fd2918fd24ed48baa4bc1ed76cdd24"}
Sep 09 12:45:28.095742 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:28.095674    2579 scope.go:117] "RemoveContainer" containerID="aaed71e59243c45d3525319172da47eb75fd1a88cfba6f71b205156c5233dcea"
Sep 09 12:45:28.103011 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:28.102982    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7801ff3-9798-4968-9fff-205f5f6c70e0" path="/var/lib/kubelet/pods/b7801ff3-9798-4968-9fff-205f5f6c70e0/volumes"
Sep 09 12:45:28.122880 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:28.122852    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"]
Sep 09 12:45:28.127937 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:28.127910    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a3f43f-5988f69787-dcrq2"]
Sep 09 12:45:30.102468 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:30.102430    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc4360ca-8972-4224-aedf-a7ee36202f36" path="/var/lib/kubelet/pods/bc4360ca-8972-4224-aedf-a7ee36202f36/volumes"
Sep 09 12:45:34.729073 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729037    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"]
Sep 09 12:45:34.729474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729373    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy"
Sep 09 12:45:34.729474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729384    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy"
Sep 09 12:45:34.729474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729396    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy"
Sep 09 12:45:34.729474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729402    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy"
Sep 09 12:45:34.729474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729428    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc4360ca-8972-4224-aedf-a7ee36202f36" containerName="payload-processing"
Sep 09 12:45:34.729474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729437    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc4360ca-8972-4224-aedf-a7ee36202f36" containerName="payload-processing"
Sep 09 12:45:34.729474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729446    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7801ff3-9798-4968-9fff-205f5f6c70e0" containerName="revoke-keys"
Sep 09 12:45:34.729474 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729452    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7801ff3-9798-4968-9fff-205f5f6c70e0" containerName="revoke-keys"
Sep 09 12:45:34.729757 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729504    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7801ff3-9798-4968-9fff-205f5f6c70e0" containerName="revoke-keys"
Sep 09 12:45:34.729757 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729512    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc4360ca-8972-4224-aedf-a7ee36202f36" containerName="payload-processing"
Sep 09 12:45:34.729757 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729518    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy"
Sep 09 12:45:34.729757 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.729524    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e88080e7-f4f7-4913-ab9b-85d173ca533f" containerName="istio-proxy"
Sep 09 12:45:34.733874 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.733856    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"
Sep 09 12:45:34.736199 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.736180    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:45:34.736327 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.736180    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:45:34.740524 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.740500    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"]
Sep 09 12:45:34.827791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.827751    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f696a-1932-4c19-bf19-54ed494b5c7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs\" (UID: \"780f696a-1932-4c19-bf19-54ed494b5c7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"
Sep 09 12:45:34.928945 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.928896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f696a-1932-4c19-bf19-54ed494b5c7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs\" (UID: \"780f696a-1932-4c19-bf19-54ed494b5c7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"
Sep 09 12:45:34.929686 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:34.929662    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f696a-1932-4c19-bf19-54ed494b5c7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs\" (UID: \"780f696a-1932-4c19-bf19-54ed494b5c7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"
Sep 09 12:45:35.045533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:35.045447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"
Sep 09 12:45:35.322454 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:35.322431    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"]
Sep 09 12:45:35.324874 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:45:35.324847    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod780f696a_1932_4c19_bf19_54ed494b5c7a.slice/crio-112369d08651a00b97c2d6c91e44f3b250ec8d10da5125d805331a9b25783b1a WatchSource:0}: Error finding container 112369d08651a00b97c2d6c91e44f3b250ec8d10da5125d805331a9b25783b1a: Status 404 returned error can't find the container with id 112369d08651a00b97c2d6c91e44f3b250ec8d10da5125d805331a9b25783b1a
Sep 09 12:45:35.353939 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:35.353914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:35.447886 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:35.447853    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:35.856276 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:35.856238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:36.135745 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:36.135710    2579 generic.go:356] "Generic (PLEG): container finished" podID="780f696a-1932-4c19-bf19-54ed494b5c7a" containerID="501f74662d8df0c5e10fc023b7f1cf5ec4871ab7efea1d6febcf758418e91d33" exitCode=0
Sep 09 12:45:36.135885 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:36.135748    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs" event={"ID":"780f696a-1932-4c19-bf19-54ed494b5c7a","Type":"ContainerDied","Data":"501f74662d8df0c5e10fc023b7f1cf5ec4871ab7efea1d6febcf758418e91d33"}
Sep 09 12:45:36.135885 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:36.135786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs" event={"ID":"780f696a-1932-4c19-bf19-54ed494b5c7a","Type":"ContainerStarted","Data":"112369d08651a00b97c2d6c91e44f3b250ec8d10da5125d805331a9b25783b1a"}
Sep 09 12:45:36.678255 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:36.678224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:37.271850 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:37.271823    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"
Sep 09 12:45:37.348178 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:37.348148    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f696a-1932-4c19-bf19-54ed494b5c7a-maas-api-service-ca\") pod \"780f696a-1932-4c19-bf19-54ed494b5c7a\" (UID: \"780f696a-1932-4c19-bf19-54ed494b5c7a\") "
Sep 09 12:45:37.348520 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:37.348500    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/780f696a-1932-4c19-bf19-54ed494b5c7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "780f696a-1932-4c19-bf19-54ed494b5c7a" (UID: "780f696a-1932-4c19-bf19-54ed494b5c7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:37.448739 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:37.448710    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/780f696a-1932-4c19-bf19-54ed494b5c7a-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:38.151593 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:38.151560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs" event={"ID":"780f696a-1932-4c19-bf19-54ed494b5c7a","Type":"ContainerDied","Data":"112369d08651a00b97c2d6c91e44f3b250ec8d10da5125d805331a9b25783b1a"}
Sep 09 12:45:38.151872 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:38.151574    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"
Sep 09 12:45:38.151872 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:38.151651    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="112369d08651a00b97c2d6c91e44f3b250ec8d10da5125d805331a9b25783b1a"
Sep 09 12:45:45.362608 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.362574    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"]
Sep 09 12:45:45.363053 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.362817    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl" podUID="8896c3c9-0832-48e8-8f16-b5354633862a" containerName="payload-processing" containerID="cri-o://19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e" gracePeriod=30
Sep 09 12:45:45.381744 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.381715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:45.480877 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.480784    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:45.541289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.541251    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"]
Sep 09 12:45:45.544319 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.544294    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-590823-rjhgs"]
Sep 09 12:45:45.550313 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.550284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:45.609689 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.609667    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:45:45.725145 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.725111    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbkd8\" (UniqueName: \"kubernetes.io/projected/8896c3c9-0832-48e8-8f16-b5354633862a-kube-api-access-sbkd8\") pod \"8896c3c9-0832-48e8-8f16-b5354633862a\" (UID: \"8896c3c9-0832-48e8-8f16-b5354633862a\") "
Sep 09 12:45:45.725339 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.725223    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8896c3c9-0832-48e8-8f16-b5354633862a-plugins-config-volume\") pod \"8896c3c9-0832-48e8-8f16-b5354633862a\" (UID: \"8896c3c9-0832-48e8-8f16-b5354633862a\") "
Sep 09 12:45:45.725700 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.725646    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8896c3c9-0832-48e8-8f16-b5354633862a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8896c3c9-0832-48e8-8f16-b5354633862a" (UID: "8896c3c9-0832-48e8-8f16-b5354633862a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:45.727818 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.727785    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8896c3c9-0832-48e8-8f16-b5354633862a-kube-api-access-sbkd8" (OuterVolumeSpecName: "kube-api-access-sbkd8") pod "8896c3c9-0832-48e8-8f16-b5354633862a" (UID: "8896c3c9-0832-48e8-8f16-b5354633862a"). InnerVolumeSpecName "kube-api-access-sbkd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:45.826265 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.826229    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8896c3c9-0832-48e8-8f16-b5354633862a-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:45.826265 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.826260    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbkd8\" (UniqueName: \"kubernetes.io/projected/8896c3c9-0832-48e8-8f16-b5354633862a-kube-api-access-sbkd8\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:45.911911 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:45.911866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:46.102167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.102088    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="780f696a-1932-4c19-bf19-54ed494b5c7a" path="/var/lib/kubelet/pods/780f696a-1932-4c19-bf19-54ed494b5c7a/volumes"
Sep 09 12:45:46.194314 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.194279    2579 generic.go:356] "Generic (PLEG): container finished" podID="8896c3c9-0832-48e8-8f16-b5354633862a" containerID="19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e" exitCode=0
Sep 09 12:45:46.194473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.194321    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"
Sep 09 12:45:46.194473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.194370    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl" event={"ID":"8896c3c9-0832-48e8-8f16-b5354633862a","Type":"ContainerDied","Data":"19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e"}
Sep 09 12:45:46.194473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.194416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl" event={"ID":"8896c3c9-0832-48e8-8f16-b5354633862a","Type":"ContainerDied","Data":"a1a22df76a8141c3f91eeaf7fda128f3063b999bfb6c5fdda5eb417d0455f53f"}
Sep 09 12:45:46.194473 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.194440    2579 scope.go:117] "RemoveContainer" containerID="19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e"
Sep 09 12:45:46.202653 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.202613    2579 scope.go:117] "RemoveContainer" containerID="19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e"
Sep 09 12:45:46.202941 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:45:46.202917    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e\": container with ID starting with 19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e not found: ID does not exist" containerID="19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e"
Sep 09 12:45:46.203034 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.202950    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e"} err="failed to get container status \"19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e\": rpc error: code = NotFound desc = could not find container \"19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e\": container with ID starting with 19b99508ea87e96914e1ab32aee6735863689131c18c339c80761c8ad28b819e not found: ID does not exist"
Sep 09 12:45:46.210487 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.210461    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"]
Sep 09 12:45:46.214769 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:46.214743    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-590823-7fb946cc6f-rs6vl"]
Sep 09 12:45:48.103173 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:48.103142    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8896c3c9-0832-48e8-8f16-b5354633862a" path="/var/lib/kubelet/pods/8896c3c9-0832-48e8-8f16-b5354633862a/volumes"
Sep 09 12:45:52.561994 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:52.561956    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:52.662680 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:52.662623    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:52.806380 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:52.806346    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"]
Sep 09 12:45:52.806689 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:52.806623    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy" containerID="cri-o://69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915" gracePeriod=30
Sep 09 12:45:52.831988 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:52.831914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:45:54.189620 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:54.189583    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:45:54.190051 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:54.189669    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:45:58.061713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.061686    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:45:58.240210 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240174    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-credential-socket\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240210 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240216    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-token\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240480 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240236    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9tfd\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-kube-api-access-r9tfd\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240480 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-data\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240480    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-data" (OuterVolumeSpecName: "istio-data") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:58.240582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240480    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-socket\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240499    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:58.240582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240541    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-podinfo\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240582 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240563    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-envoy\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240884 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240588    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istiod-ca-cert\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240884 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240696    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-certs\") pod \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\" (UID: \"33b633ad-9f34-4346-baa1-e3ec33ef7c30\") "
Sep 09 12:45:58.240884 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240711    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:58.241059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.240995    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-data\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:58.241059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.241014    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:58.241059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.241028    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-credential-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:58.241059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.241004    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:58.241059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.241027    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:45:58.242503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.242476    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-token" (OuterVolumeSpecName: "istio-token") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:58.242648 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.242617    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:45:58.242794 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.242773    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-kube-api-access-r9tfd" (OuterVolumeSpecName: "kube-api-access-r9tfd") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "kube-api-access-r9tfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:45:58.242840 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.242817    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33b633ad-9f34-4346-baa1-e3ec33ef7c30" (UID: "33b633ad-9f34-4346-baa1-e3ec33ef7c30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:45:58.261903 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.261873    2579 generic.go:356] "Generic (PLEG): container finished" podID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerID="69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915" exitCode=0
Sep 09 12:45:58.262034 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.261912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" event={"ID":"33b633ad-9f34-4346-baa1-e3ec33ef7c30","Type":"ContainerDied","Data":"69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915"}
Sep 09 12:45:58.262034 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.261953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz" event={"ID":"33b633ad-9f34-4346-baa1-e3ec33ef7c30","Type":"ContainerDied","Data":"f3583e2f2dd665c164c88660ef29dd760694fc3aa3ab6a846d11bbd98aa5dd76"}
Sep 09 12:45:58.262034 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.261976    2579 scope.go:117] "RemoveContainer" containerID="69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915"
Sep 09 12:45:58.262034 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.261913    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"
Sep 09 12:45:58.273884 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.273862    2579 scope.go:117] "RemoveContainer" containerID="a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263"
Sep 09 12:45:58.281799 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.281777    2579 scope.go:117] "RemoveContainer" containerID="69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915"
Sep 09 12:45:58.282098 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:45:58.282075    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915\": container with ID starting with 69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915 not found: ID does not exist" containerID="69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915"
Sep 09 12:45:58.282178 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.282111    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915"} err="failed to get container status \"69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915\": rpc error: code = NotFound desc = could not find container \"69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915\": container with ID starting with 69fc7d9393c6ddb110a6898445eaf0281943b375e2d61016f7e25dffb6be2915 not found: ID does not exist"
Sep 09 12:45:58.282178 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.282132    2579 scope.go:117] "RemoveContainer" containerID="a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263"
Sep 09 12:45:58.282509 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:45:58.282483    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263\": container with ID starting with a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263 not found: ID does not exist" containerID="a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263"
Sep 09 12:45:58.282600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.282517    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263"} err="failed to get container status \"a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263\": rpc error: code = NotFound desc = could not find container \"a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263\": container with ID starting with a157d829c9a3b2007b71eb3ac3f73cf3cd0c43abcba69971cc0d68c68ec36263 not found: ID does not exist"
Sep 09 12:45:58.284695 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.284675    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"]
Sep 09 12:45:58.288510 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.288491    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-590823-openshift-default-6cb6c9c9c4-p76rz"]
Sep 09 12:45:58.342451 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.342411    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-token\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:58.342451 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.342449    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9tfd\" (UniqueName: \"kubernetes.io/projected/33b633ad-9f34-4346-baa1-e3ec33ef7c30-kube-api-access-r9tfd\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:58.342451 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.342461    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-podinfo\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:58.342692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.342471    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istio-envoy\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:58.342692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.342479    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b633ad-9f34-4346-baa1-e3ec33ef7c30-istiod-ca-cert\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:45:58.342692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:45:58.342486    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b633ad-9f34-4346-baa1-e3ec33ef7c30-workload-certs\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:46:00.102622 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:00.102573    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" path="/var/lib/kubelet/pods/33b633ad-9f34-4346-baa1-e3ec33ef7c30/volumes"
Sep 09 12:46:02.421781 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:02.421732    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:03.483050 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:03.483008    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:03.916004 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:03.915887    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:04.249071 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:04.249029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:04.941114 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:04.941073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:07.052361 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:07.052331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:07.151846 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:07.151807    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:08.364940 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:08.364879    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:08.465413 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:08.465380    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:08.566226 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:08.566177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:08.954043 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:08.954002    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:09.046722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.046690    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:09.053915 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.053884    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"]
Sep 09 12:46:09.054275 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054258    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="780f696a-1932-4c19-bf19-54ed494b5c7a" containerName="revoke-keys"
Sep 09 12:46:09.054364 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054277    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="780f696a-1932-4c19-bf19-54ed494b5c7a" containerName="revoke-keys"
Sep 09 12:46:09.054364 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054295    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy"
Sep 09 12:46:09.054364 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054303    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy"
Sep 09 12:46:09.054364 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054316    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8896c3c9-0832-48e8-8f16-b5354633862a" containerName="payload-processing"
Sep 09 12:46:09.054364 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054324    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8896c3c9-0832-48e8-8f16-b5354633862a" containerName="payload-processing"
Sep 09 12:46:09.054617 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054421    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="780f696a-1932-4c19-bf19-54ed494b5c7a" containerName="revoke-keys"
Sep 09 12:46:09.054617 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054436    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy"
Sep 09 12:46:09.054617 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054449    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8896c3c9-0832-48e8-8f16-b5354633862a" containerName="payload-processing"
Sep 09 12:46:09.054617 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.054457    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy"
Sep 09 12:46:09.058864 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.058843    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:09.061027 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.061010    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-610d01\""
Sep 09 12:46:09.061134 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.061012    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-610d01-dockercfg-gqkpl\""
Sep 09 12:46:09.065010 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.064987    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"]
Sep 09 12:46:09.068143 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.068121    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"]
Sep 09 12:46:09.068470 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.068457    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy"
Sep 09 12:46:09.068535 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.068471    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b633ad-9f34-4346-baa1-e3ec33ef7c30" containerName="istio-proxy"
Sep 09 12:46:09.071304 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.071288    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:09.090254 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.090227    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"]
Sep 09 12:46:09.123245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.123206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl\" (UID: \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:09.123411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.123253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh7sf\" (UniqueName: \"kubernetes.io/projected/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-kube-api-access-xh7sf\") pod \"payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg\" (UID: \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:09.123411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.123325    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg\" (UID: \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:09.123411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.123382    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sshk4\" (UniqueName: \"kubernetes.io/projected/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-kube-api-access-sshk4\") pod \"payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl\" (UID: \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:09.224765 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.224686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg\" (UID: \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:09.224765 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.224740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sshk4\" (UniqueName: \"kubernetes.io/projected/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-kube-api-access-sshk4\") pod \"payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl\" (UID: \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:09.224957 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.224793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl\" (UID: \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:09.224957 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.224822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh7sf\" (UniqueName: \"kubernetes.io/projected/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-kube-api-access-xh7sf\") pod \"payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg\" (UID: \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:09.225610 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.225586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg\" (UID: \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:09.225683 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.225586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl\" (UID: \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:09.232893 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.232863    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh7sf\" (UniqueName: \"kubernetes.io/projected/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-kube-api-access-xh7sf\") pod \"payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg\" (UID: \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:09.233012 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.232863    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sshk4\" (UniqueName: \"kubernetes.io/projected/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-kube-api-access-sshk4\") pod \"payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl\" (UID: \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:09.369867 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.369837    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:09.388751 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.388725    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:09.641483 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.641456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"]
Sep 09 12:46:09.642876 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:46:09.642847    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12153e69_72a7_4bc2_b7b4_08744d8b5c2a.slice/crio-a094a916b8e38264bf301c953a519ddc444e67178e6dac2ddabb7bb37cc8cd62 WatchSource:0}: Error finding container a094a916b8e38264bf301c953a519ddc444e67178e6dac2ddabb7bb37cc8cd62: Status 404 returned error can't find the container with id a094a916b8e38264bf301c953a519ddc444e67178e6dac2ddabb7bb37cc8cd62
Sep 09 12:46:09.668606 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:09.668577    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"]
Sep 09 12:46:09.670922 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:46:09.670898    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8fd1789_891d_44f6_b16a_71ab32ec4ae0.slice/crio-7616ff1e03e609c6f056e98120430ee076d864666c631446ab72e3b9b2ad8228 WatchSource:0}: Error finding container 7616ff1e03e609c6f056e98120430ee076d864666c631446ab72e3b9b2ad8228: Status 404 returned error can't find the container with id 7616ff1e03e609c6f056e98120430ee076d864666c631446ab72e3b9b2ad8228
Sep 09 12:46:10.324090 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:10.324048    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg" event={"ID":"12153e69-72a7-4bc2-b7b4-08744d8b5c2a","Type":"ContainerStarted","Data":"7d879971bf22b023d2b2c0ae4a090b6d31f4627f128ddf8c19a90faaa996d269"}
Sep 09 12:46:10.324302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:10.324112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg" event={"ID":"12153e69-72a7-4bc2-b7b4-08744d8b5c2a","Type":"ContainerStarted","Data":"a094a916b8e38264bf301c953a519ddc444e67178e6dac2ddabb7bb37cc8cd62"}
Sep 09 12:46:10.324302 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:10.324139    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:10.325657 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:10.325612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl" event={"ID":"d8fd1789-891d-44f6-b16a-71ab32ec4ae0","Type":"ContainerStarted","Data":"e93c5096fd879842078345bab3594c2c0533f99ce8e5f4297faedf6e3310a7b0"}
Sep 09 12:46:10.325764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:10.325671    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl" event={"ID":"d8fd1789-891d-44f6-b16a-71ab32ec4ae0","Type":"ContainerStarted","Data":"7616ff1e03e609c6f056e98120430ee076d864666c631446ab72e3b9b2ad8228"}
Sep 09 12:46:10.325764 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:10.325711    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:10.341270 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:10.341224    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg" podStartSLOduration=1.341209889 podStartE2EDuration="1.341209889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:46:10.339661298 +0000 UTC m=+1586.663881827" watchObservedRunningTime="2026-09-09 12:46:10.341209889 +0000 UTC m=+1586.665430420"
Sep 09 12:46:10.356252 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:10.356203    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl" podStartSLOduration=1.356189477 podStartE2EDuration="1.356189477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:46:10.354007009 +0000 UTC m=+1586.678227544" watchObservedRunningTime="2026-09-09 12:46:10.356189477 +0000 UTC m=+1586.680410008"
Sep 09 12:46:21.333467 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:21.333429    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:46:21.333937 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:21.333512    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:46:32.098002 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:32.097966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:32.799653 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:32.799610    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:33.055735 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.055655    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"]
Sep 09 12:46:33.059105 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.059080    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.061187 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.061162    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-2db0a6-openshift-default-dockercfg-8vthm\""
Sep 09 12:46:33.072403 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.072376    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"]
Sep 09 12:46:33.096258 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.096228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:33.132380 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-credential-socket\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.132791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4743e947-fe49-4b95-9346-92879a434e40-istiod-ca-cert\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.132791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-envoy\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.132791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132477    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrmpw\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-kube-api-access-qrmpw\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.132791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132529    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-data\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.132791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-certs\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.132791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-istio-token\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.132791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-socket\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.132791 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.132608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4743e947-fe49-4b95-9346-92879a434e40-istio-podinfo\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233291 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-envoy\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233291 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrmpw\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-kube-api-access-qrmpw\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-data\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-certs\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-istio-token\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-socket\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233533 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4743e947-fe49-4b95-9346-92879a434e40-istio-podinfo\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233852 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-credential-socket\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233852 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4743e947-fe49-4b95-9346-92879a434e40-istiod-ca-cert\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.233852 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.233818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-certs\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.234212 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.234186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-credential-socket\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.234317 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.234239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-data\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.234317 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.234256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-socket\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.234440 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.234331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4743e947-fe49-4b95-9346-92879a434e40-istiod-ca-cert\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.235863 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.235837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-envoy\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.236027 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.236011    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4743e947-fe49-4b95-9346-92879a434e40-istio-podinfo\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.240986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.240964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-istio-token\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.241163 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.241144    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrmpw\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-kube-api-access-qrmpw\") pod \"e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") " pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.371546 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.371458    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:33.428363 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.428333    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:33.647631 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.647604    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"]
Sep 09 12:46:33.649094 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:46:33.649069    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4743e947_fe49_4b95_9346_92879a434e40.slice/crio-e110a45d23b5b214f20666c8ca7e7614014e0fe7e83301e7590bc34a4003c1c3 WatchSource:0}: Error finding container e110a45d23b5b214f20666c8ca7e7614014e0fe7e83301e7590bc34a4003c1c3: Status 404 returned error can't find the container with id e110a45d23b5b214f20666c8ca7e7614014e0fe7e83301e7590bc34a4003c1c3
Sep 09 12:46:33.651353 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.651324    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:46:33.651426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.651388    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:46:33.651426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:33.651413    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:46:34.451537 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:34.451499    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" event={"ID":"4743e947-fe49-4b95-9346-92879a434e40","Type":"ContainerStarted","Data":"185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49"}
Sep 09 12:46:34.451951 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:34.451584    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" event={"ID":"4743e947-fe49-4b95-9346-92879a434e40","Type":"ContainerStarted","Data":"e110a45d23b5b214f20666c8ca7e7614014e0fe7e83301e7590bc34a4003c1c3"}
Sep 09 12:46:34.470275 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:34.470216    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" podStartSLOduration=1.470196902 podStartE2EDuration="1.470196902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:46:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:46:34.469047603 +0000 UTC m=+1610.793268149" watchObservedRunningTime="2026-09-09 12:46:34.470196902 +0000 UTC m=+1610.794417430"
Sep 09 12:46:35.372037 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:35.371997    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:35.377222 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:35.377194    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:35.456548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:35.456507    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:35.457490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:35.457471    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:36.391574 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:36.391459    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:36.492245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:36.492210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:37.632206 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:37.632169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:37.733282 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:37.733247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:38.163915 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.163888    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:38.252693 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.252658    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"]
Sep 09 12:46:38.255934 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.255917    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:38.258084 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.258060    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2db0a6\""
Sep 09 12:46:38.258204 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.258115    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2db0a6-dockercfg-ppf4p\""
Sep 09 12:46:38.267611 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.267567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"]
Sep 09 12:46:38.317254 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.317218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:46:38.378378 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.378340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4jfs\" (UniqueName: \"kubernetes.io/projected/9eaa666b-80b4-4c94-978b-7dccca53990f-kube-api-access-l4jfs\") pod \"payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t\" (UID: \"9eaa666b-80b4-4c94-978b-7dccca53990f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:38.378541 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.378418    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eaa666b-80b4-4c94-978b-7dccca53990f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t\" (UID: \"9eaa666b-80b4-4c94-978b-7dccca53990f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:38.479770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.479728    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4jfs\" (UniqueName: \"kubernetes.io/projected/9eaa666b-80b4-4c94-978b-7dccca53990f-kube-api-access-l4jfs\") pod \"payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t\" (UID: \"9eaa666b-80b4-4c94-978b-7dccca53990f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:38.479965 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.479811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eaa666b-80b4-4c94-978b-7dccca53990f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t\" (UID: \"9eaa666b-80b4-4c94-978b-7dccca53990f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:38.480518 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.480496    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eaa666b-80b4-4c94-978b-7dccca53990f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t\" (UID: \"9eaa666b-80b4-4c94-978b-7dccca53990f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:38.487320 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.487291    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4jfs\" (UniqueName: \"kubernetes.io/projected/9eaa666b-80b4-4c94-978b-7dccca53990f-kube-api-access-l4jfs\") pod \"payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t\" (UID: \"9eaa666b-80b4-4c94-978b-7dccca53990f\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:38.569313 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.569269    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:38.838745 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:38.838690    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"]
Sep 09 12:46:38.841412 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:46:38.841383    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eaa666b_80b4_4c94_978b_7dccca53990f.slice/crio-a89a343a91f2d88cf8c76bcaed652a8f2774fcf0eac6d92f63f6cdafe1eb5705 WatchSource:0}: Error finding container a89a343a91f2d88cf8c76bcaed652a8f2774fcf0eac6d92f63f6cdafe1eb5705: Status 404 returned error can't find the container with id a89a343a91f2d88cf8c76bcaed652a8f2774fcf0eac6d92f63f6cdafe1eb5705
Sep 09 12:46:39.478501 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:39.478461    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t" event={"ID":"9eaa666b-80b4-4c94-978b-7dccca53990f","Type":"ContainerStarted","Data":"1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709"}
Sep 09 12:46:39.478729 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:39.478526    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t" event={"ID":"9eaa666b-80b4-4c94-978b-7dccca53990f","Type":"ContainerStarted","Data":"a89a343a91f2d88cf8c76bcaed652a8f2774fcf0eac6d92f63f6cdafe1eb5705"}
Sep 09 12:46:39.478729 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:39.478559    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:39.494439 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:39.494391    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t" podStartSLOduration=1.494376189 podStartE2EDuration="1.494376189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:46:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:46:39.493626838 +0000 UTC m=+1615.817847371" watchObservedRunningTime="2026-09-09 12:46:39.494376189 +0000 UTC m=+1615.818596719"
Sep 09 12:46:50.458614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.458576    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:46:50.459049 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.458676    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 09 12:46:50.486800 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.486774    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:46:50.536810 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.536772    2579 generic.go:356] "Generic (PLEG): container finished" podID="4743e947-fe49-4b95-9346-92879a434e40" containerID="185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49" exitCode=0
Sep 09 12:46:50.536978 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.536831    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" event={"ID":"4743e947-fe49-4b95-9346-92879a434e40","Type":"ContainerDied","Data":"185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49"}
Sep 09 12:46:50.537200 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.537185    2579 scope.go:117] "RemoveContainer" containerID="185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49"
Sep 09 12:46:50.538325 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.538299    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:46:50.538387 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.538369    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:46:50.538423 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:50.538405    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 12:46:51.543613 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:51.543578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" event={"ID":"4743e947-fe49-4b95-9346-92879a434e40","Type":"ContainerStarted","Data":"9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893"}
Sep 09 12:46:52.372427 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:52.372383    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:52.372766 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:52.372661    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:46:52.372766 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:52.372731    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 09 12:46:53.372012 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:53.371973    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:46:53.372410 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:53.372028    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 09 12:46:54.371980 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:54.371939    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:46:54.372163 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:54.372004    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 09 12:46:55.372851 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:55.372803    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:46:55.373349 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:55.372874    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 09 12:46:56.372107 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:56.372066    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 09 12:46:56.372298 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:56.372125    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 09 12:46:57.376178 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:57.376142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:57.376664 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:57.376423    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:46:57.377111 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:46:57.377090    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:47:01.421117 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:01.421078    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:06.608823 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:06.608781    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t_9eaa666b-80b4-4c94-978b-7dccca53990f/payload-processing/0.log"
Sep 09 12:47:11.151174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:11.151140    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:34.148696 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:34.148666    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl_d8fd1789-891d-44f6-b16a-71ab32ec4ae0/payload-processing/0.log"
Sep 09 12:47:34.405476 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:34.405391    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t_9eaa666b-80b4-4c94-978b-7dccca53990f/payload-processing/0.log"
Sep 09 12:47:34.718726 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:34.718694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:35.054496 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:35.054404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:35.232086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:35.232047    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:38.680750 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:38.680718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:38.781329 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:38.781292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:39.857301 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:39.857265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:39.958674 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:39.958628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:40.075849 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.075816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:40.358564 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.358530    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:40.451157 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.451125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:47:40.478855 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.478825    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"]
Sep 09 12:47:40.482207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.482189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:40.484301 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.484272    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-90dee1-dockercfg-92mxn\""
Sep 09 12:47:40.484434 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.484305    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-90dee1\""
Sep 09 12:47:40.491648 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.491609    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"]
Sep 09 12:47:40.611396 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.611306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4htlw\" (UniqueName: \"kubernetes.io/projected/d4437e67-632d-4ffb-9f0d-ad48950859fb-kube-api-access-4htlw\") pod \"payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4\" (UID: \"d4437e67-632d-4ffb-9f0d-ad48950859fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:40.611396 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.611358    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4437e67-632d-4ffb-9f0d-ad48950859fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4\" (UID: \"d4437e67-632d-4ffb-9f0d-ad48950859fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:40.712480 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.712444    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4437e67-632d-4ffb-9f0d-ad48950859fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4\" (UID: \"d4437e67-632d-4ffb-9f0d-ad48950859fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:40.712663 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.712606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4htlw\" (UniqueName: \"kubernetes.io/projected/d4437e67-632d-4ffb-9f0d-ad48950859fb-kube-api-access-4htlw\") pod \"payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4\" (UID: \"d4437e67-632d-4ffb-9f0d-ad48950859fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:40.713173 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.713152    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4437e67-632d-4ffb-9f0d-ad48950859fb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4\" (UID: \"d4437e67-632d-4ffb-9f0d-ad48950859fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:40.720704 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.720677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4htlw\" (UniqueName: \"kubernetes.io/projected/d4437e67-632d-4ffb-9f0d-ad48950859fb-kube-api-access-4htlw\") pod \"payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4\" (UID: \"d4437e67-632d-4ffb-9f0d-ad48950859fb\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:40.793457 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:40.793425    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:41.062943 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:41.062906    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"]
Sep 09 12:47:41.065968 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:47:41.065936    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4437e67_632d_4ffb_9f0d_ad48950859fb.slice/crio-2f79f4cf078837cc7b3078d94acbadcac8f26c33f226fa23da9516f69458969c WatchSource:0}: Error finding container 2f79f4cf078837cc7b3078d94acbadcac8f26c33f226fa23da9516f69458969c: Status 404 returned error can't find the container with id 2f79f4cf078837cc7b3078d94acbadcac8f26c33f226fa23da9516f69458969c
Sep 09 12:47:41.807142 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:41.807106    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4" event={"ID":"d4437e67-632d-4ffb-9f0d-ad48950859fb","Type":"ContainerStarted","Data":"371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344"}
Sep 09 12:47:41.807330 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:41.807160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4" event={"ID":"d4437e67-632d-4ffb-9f0d-ad48950859fb","Type":"ContainerStarted","Data":"2f79f4cf078837cc7b3078d94acbadcac8f26c33f226fa23da9516f69458969c"}
Sep 09 12:47:41.807330 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:41.807250    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:47:41.824149 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:41.824093    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4" podStartSLOduration=1.8240752580000001 podStartE2EDuration="1.824075258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:47:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:47:41.823895733 +0000 UTC m=+1678.148116264" watchObservedRunningTime="2026-09-09 12:47:41.824075258 +0000 UTC m=+1678.148295792"
Sep 09 12:47:44.381078 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:44.381042    2579 scope.go:117] "RemoveContainer" containerID="ddac718786a19ba5c95a1bcb2ea10d08f637bebd2866177d4320623f4f5bb86d"
Sep 09 12:47:52.815832 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:47:52.815798    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:48:03.739400 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:03.739366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:04.433577 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.433544    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"]
Sep 09 12:48:04.436977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.436962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"
Sep 09 12:48:04.439031 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.439006    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:48:04.439157 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.439107    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:48:04.444252 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.443871    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"]
Sep 09 12:48:04.504208 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.504181    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efd24c88-7088-42e5-b4f1-af67e9d910bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48\" (UID: \"efd24c88-7088-42e5-b4f1-af67e9d910bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"
Sep 09 12:48:04.605560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.605527    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efd24c88-7088-42e5-b4f1-af67e9d910bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48\" (UID: \"efd24c88-7088-42e5-b4f1-af67e9d910bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"
Sep 09 12:48:04.606172 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.606153    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efd24c88-7088-42e5-b4f1-af67e9d910bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48\" (UID: \"efd24c88-7088-42e5-b4f1-af67e9d910bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"
Sep 09 12:48:04.748282 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:04.748246    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"
Sep 09 12:48:05.020241 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:05.020133    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"]
Sep 09 12:48:05.025680 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:48:05.025652    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd24c88_7088_42e5_b4f1_af67e9d910bf.slice/crio-f0ee2e019965f64e87fa344118a253bbb2b1af350635d34de6d3fd6e75b4d848 WatchSource:0}: Error finding container f0ee2e019965f64e87fa344118a253bbb2b1af350635d34de6d3fd6e75b4d848: Status 404 returned error can't find the container with id f0ee2e019965f64e87fa344118a253bbb2b1af350635d34de6d3fd6e75b4d848
Sep 09 12:48:05.930249 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:05.930212    2579 generic.go:356] "Generic (PLEG): container finished" podID="efd24c88-7088-42e5-b4f1-af67e9d910bf" containerID="4ddce4a2f74a1f546ccf42606605ba4af191cb414b3055aa6f5ef58a033b624a" exitCode=0
Sep 09 12:48:05.930615 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:05.930258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48" event={"ID":"efd24c88-7088-42e5-b4f1-af67e9d910bf","Type":"ContainerDied","Data":"4ddce4a2f74a1f546ccf42606605ba4af191cb414b3055aa6f5ef58a033b624a"}
Sep 09 12:48:05.930615 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:05.930311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48" event={"ID":"efd24c88-7088-42e5-b4f1-af67e9d910bf","Type":"ContainerStarted","Data":"f0ee2e019965f64e87fa344118a253bbb2b1af350635d34de6d3fd6e75b4d848"}
Sep 09 12:48:07.062442 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:07.062415    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"
Sep 09 12:48:07.127522 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:07.127484    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efd24c88-7088-42e5-b4f1-af67e9d910bf-maas-api-service-ca\") pod \"efd24c88-7088-42e5-b4f1-af67e9d910bf\" (UID: \"efd24c88-7088-42e5-b4f1-af67e9d910bf\") "
Sep 09 12:48:07.127895 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:07.127873    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efd24c88-7088-42e5-b4f1-af67e9d910bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efd24c88-7088-42e5-b4f1-af67e9d910bf" (UID: "efd24c88-7088-42e5-b4f1-af67e9d910bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:48:07.228396 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:07.228367    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efd24c88-7088-42e5-b4f1-af67e9d910bf-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:48:07.941709 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:07.941670    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48" event={"ID":"efd24c88-7088-42e5-b4f1-af67e9d910bf","Type":"ContainerDied","Data":"f0ee2e019965f64e87fa344118a253bbb2b1af350635d34de6d3fd6e75b4d848"}
Sep 09 12:48:07.941709 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:07.941692    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"
Sep 09 12:48:07.941933 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:07.941730    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0ee2e019965f64e87fa344118a253bbb2b1af350635d34de6d3fd6e75b4d848"
Sep 09 12:48:10.360036 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.359999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:10.393137 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.393098    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"]
Sep 09 12:48:10.393426 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.393398    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4" podUID="d4437e67-632d-4ffb-9f0d-ad48950859fb" containerName="payload-processing" containerID="cri-o://371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344" gracePeriod=30
Sep 09 12:48:10.460550 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.460511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:10.560268 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.560235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:10.566507 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.566460    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"]
Sep 09 12:48:10.568399 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.568377    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-90dee1-gst48"]
Sep 09 12:48:10.659561 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.659490    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:48:10.660863 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.660828    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:10.757561 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.757523    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4htlw\" (UniqueName: \"kubernetes.io/projected/d4437e67-632d-4ffb-9f0d-ad48950859fb-kube-api-access-4htlw\") pod \"d4437e67-632d-4ffb-9f0d-ad48950859fb\" (UID: \"d4437e67-632d-4ffb-9f0d-ad48950859fb\") "
Sep 09 12:48:10.757788 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.757766    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4437e67-632d-4ffb-9f0d-ad48950859fb-plugins-config-volume\") pod \"d4437e67-632d-4ffb-9f0d-ad48950859fb\" (UID: \"d4437e67-632d-4ffb-9f0d-ad48950859fb\") "
Sep 09 12:48:10.758092 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.758069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4437e67-632d-4ffb-9f0d-ad48950859fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4437e67-632d-4ffb-9f0d-ad48950859fb" (UID: "d4437e67-632d-4ffb-9f0d-ad48950859fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:48:10.759658 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.759620    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4437e67-632d-4ffb-9f0d-ad48950859fb-kube-api-access-4htlw" (OuterVolumeSpecName: "kube-api-access-4htlw") pod "d4437e67-632d-4ffb-9f0d-ad48950859fb" (UID: "d4437e67-632d-4ffb-9f0d-ad48950859fb"). InnerVolumeSpecName "kube-api-access-4htlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:48:10.858873 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.858830    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4437e67-632d-4ffb-9f0d-ad48950859fb-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:48:10.858873 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.858865    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4htlw\" (UniqueName: \"kubernetes.io/projected/d4437e67-632d-4ffb-9f0d-ad48950859fb-kube-api-access-4htlw\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:48:10.936375 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.936296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:10.958425 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.958388    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4437e67-632d-4ffb-9f0d-ad48950859fb" containerID="371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344" exitCode=0
Sep 09 12:48:10.958569 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.958436    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"
Sep 09 12:48:10.958569 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.958472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4" event={"ID":"d4437e67-632d-4ffb-9f0d-ad48950859fb","Type":"ContainerDied","Data":"371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344"}
Sep 09 12:48:10.958569 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.958525    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4" event={"ID":"d4437e67-632d-4ffb-9f0d-ad48950859fb","Type":"ContainerDied","Data":"2f79f4cf078837cc7b3078d94acbadcac8f26c33f226fa23da9516f69458969c"}
Sep 09 12:48:10.958569 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.958548    2579 scope.go:117] "RemoveContainer" containerID="371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344"
Sep 09 12:48:10.967436 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.967416    2579 scope.go:117] "RemoveContainer" containerID="371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344"
Sep 09 12:48:10.967744 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:48:10.967718    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344\": container with ID starting with 371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344 not found: ID does not exist" containerID="371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344"
Sep 09 12:48:10.967845 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.967757    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344"} err="failed to get container status \"371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344\": rpc error: code = NotFound desc = could not find container \"371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344\": container with ID starting with 371da0dea1609ebc5793b2c4fc4228ee8aaa1efd1da2e9cc4a2e560d2fc61344 not found: ID does not exist"
Sep 09 12:48:10.980546 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.980518    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"]
Sep 09 12:48:10.982351 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:10.982330    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-90dee1-647b8884c-q2gp4"]
Sep 09 12:48:12.102937 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:12.102903    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4437e67-632d-4ffb-9f0d-ad48950859fb" path="/var/lib/kubelet/pods/d4437e67-632d-4ffb-9f0d-ad48950859fb/volumes"
Sep 09 12:48:12.103328 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:12.103231    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efd24c88-7088-42e5-b4f1-af67e9d910bf" path="/var/lib/kubelet/pods/efd24c88-7088-42e5-b4f1-af67e9d910bf/volumes"
Sep 09 12:48:17.425342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:17.425308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:17.526161 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:17.526129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:17.697808 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:17.697729    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:21.371781 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.371747    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"]
Sep 09 12:48:21.372236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.372106    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efd24c88-7088-42e5-b4f1-af67e9d910bf" containerName="revoke-keys"
Sep 09 12:48:21.372236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.372118    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="efd24c88-7088-42e5-b4f1-af67e9d910bf" containerName="revoke-keys"
Sep 09 12:48:21.372236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.372127    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4437e67-632d-4ffb-9f0d-ad48950859fb" containerName="payload-processing"
Sep 09 12:48:21.372236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.372133    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4437e67-632d-4ffb-9f0d-ad48950859fb" containerName="payload-processing"
Sep 09 12:48:21.372236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.372188    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="efd24c88-7088-42e5-b4f1-af67e9d910bf" containerName="revoke-keys"
Sep 09 12:48:21.372236 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.372197    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4437e67-632d-4ffb-9f0d-ad48950859fb" containerName="payload-processing"
Sep 09 12:48:21.375039 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.375021    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"
Sep 09 12:48:21.376978 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.376957    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:48:21.377086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.376958    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:48:21.382555 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.382471    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"]
Sep 09 12:48:21.450224 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.450166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74cfcfce-882f-46be-bce9-750fe421f0a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2\" (UID: \"74cfcfce-882f-46be-bce9-750fe421f0a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"
Sep 09 12:48:21.551437 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.551387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74cfcfce-882f-46be-bce9-750fe421f0a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2\" (UID: \"74cfcfce-882f-46be-bce9-750fe421f0a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"
Sep 09 12:48:21.552056 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.552027    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74cfcfce-882f-46be-bce9-750fe421f0a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2\" (UID: \"74cfcfce-882f-46be-bce9-750fe421f0a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"
Sep 09 12:48:21.632516 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.632410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:21.686487 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.686454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"
Sep 09 12:48:21.961289 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:21.960774    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"]
Sep 09 12:48:22.022644 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:22.022609    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2" event={"ID":"74cfcfce-882f-46be-bce9-750fe421f0a3","Type":"ContainerStarted","Data":"83acf1d1da8bb3a7fea11b394280e1bf68876130a8eba4c006a313cd5e07db9e"}
Sep 09 12:48:23.029396 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:23.029359    2579 generic.go:356] "Generic (PLEG): container finished" podID="74cfcfce-882f-46be-bce9-750fe421f0a3" containerID="eb833d50ad5f5c69cdd009b4a1fd84aec2325cae877da77cd8b8d70be8eba22e" exitCode=0
Sep 09 12:48:23.029854 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:23.029438    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2" event={"ID":"74cfcfce-882f-46be-bce9-750fe421f0a3","Type":"ContainerDied","Data":"eb833d50ad5f5c69cdd009b4a1fd84aec2325cae877da77cd8b8d70be8eba22e"}
Sep 09 12:48:24.162303 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:24.162277    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"
Sep 09 12:48:24.275713 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:24.275677    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74cfcfce-882f-46be-bce9-750fe421f0a3-maas-api-service-ca\") pod \"74cfcfce-882f-46be-bce9-750fe421f0a3\" (UID: \"74cfcfce-882f-46be-bce9-750fe421f0a3\") "
Sep 09 12:48:24.276053 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:24.276028    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74cfcfce-882f-46be-bce9-750fe421f0a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "74cfcfce-882f-46be-bce9-750fe421f0a3" (UID: "74cfcfce-882f-46be-bce9-750fe421f0a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:48:24.377234 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:24.377150    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74cfcfce-882f-46be-bce9-750fe421f0a3-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:48:25.040935 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:25.040901    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"
Sep 09 12:48:25.041123 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:25.040941    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2" event={"ID":"74cfcfce-882f-46be-bce9-750fe421f0a3","Type":"ContainerDied","Data":"83acf1d1da8bb3a7fea11b394280e1bf68876130a8eba4c006a313cd5e07db9e"}
Sep 09 12:48:25.041123 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:25.040998    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83acf1d1da8bb3a7fea11b394280e1bf68876130a8eba4c006a313cd5e07db9e"
Sep 09 12:48:36.944701 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:36.944667    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"]
Sep 09 12:48:36.945158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:36.944920    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl" podUID="d8fd1789-891d-44f6-b16a-71ab32ec4ae0" containerName="payload-processing" containerID="cri-o://e93c5096fd879842078345bab3594c2c0533f99ce8e5f4297faedf6e3310a7b0" gracePeriod=30
Sep 09 12:48:36.955833 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:36.955804    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"]
Sep 09 12:48:36.956076 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:36.956029    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg" podUID="12153e69-72a7-4bc2-b7b4-08744d8b5c2a" containerName="payload-pre-processing" containerID="cri-o://7d879971bf22b023d2b2c0ae4a090b6d31f4627f128ddf8c19a90faaa996d269" gracePeriod=30
Sep 09 12:48:36.988331 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:36.988293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:37.107972 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.107938    2579 generic.go:356] "Generic (PLEG): container finished" podID="12153e69-72a7-4bc2-b7b4-08744d8b5c2a" containerID="7d879971bf22b023d2b2c0ae4a090b6d31f4627f128ddf8c19a90faaa996d269" exitCode=0
Sep 09 12:48:37.108168 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.108011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg" event={"ID":"12153e69-72a7-4bc2-b7b4-08744d8b5c2a","Type":"ContainerDied","Data":"7d879971bf22b023d2b2c0ae4a090b6d31f4627f128ddf8c19a90faaa996d269"}
Sep 09 12:48:37.110079 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.109877    2579 generic.go:356] "Generic (PLEG): container finished" podID="d8fd1789-891d-44f6-b16a-71ab32ec4ae0" containerID="e93c5096fd879842078345bab3594c2c0533f99ce8e5f4297faedf6e3310a7b0" exitCode=0
Sep 09 12:48:37.110079 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.109954    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl" event={"ID":"d8fd1789-891d-44f6-b16a-71ab32ec4ae0","Type":"ContainerDied","Data":"e93c5096fd879842078345bab3594c2c0533f99ce8e5f4297faedf6e3310a7b0"}
Sep 09 12:48:37.113491 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.113446    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"]
Sep 09 12:48:37.115190 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.115169    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-610d01-6kvl2"]
Sep 09 12:48:37.128885 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.128845    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:37.214625 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.214587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:37.233345 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.233324    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:48:37.235876 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.235858    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:48:37.398095 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.398062    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sshk4\" (UniqueName: \"kubernetes.io/projected/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-kube-api-access-sshk4\") pod \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\" (UID: \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\") "
Sep 09 12:48:37.398307 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.398108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-plugins-config-volume\") pod \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\" (UID: \"d8fd1789-891d-44f6-b16a-71ab32ec4ae0\") "
Sep 09 12:48:37.398307 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.398206    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-plugins-config-volume\") pod \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\" (UID: \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\") "
Sep 09 12:48:37.398307 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.398287    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh7sf\" (UniqueName: \"kubernetes.io/projected/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-kube-api-access-xh7sf\") pod \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\" (UID: \"12153e69-72a7-4bc2-b7b4-08744d8b5c2a\") "
Sep 09 12:48:37.398526 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.398497    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12153e69-72a7-4bc2-b7b4-08744d8b5c2a" (UID: "12153e69-72a7-4bc2-b7b4-08744d8b5c2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:48:37.398526 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.398508    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d8fd1789-891d-44f6-b16a-71ab32ec4ae0" (UID: "d8fd1789-891d-44f6-b16a-71ab32ec4ae0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:48:37.400339 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.400310    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-kube-api-access-xh7sf" (OuterVolumeSpecName: "kube-api-access-xh7sf") pod "12153e69-72a7-4bc2-b7b4-08744d8b5c2a" (UID: "12153e69-72a7-4bc2-b7b4-08744d8b5c2a"). InnerVolumeSpecName "kube-api-access-xh7sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:48:37.400446 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.400365    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-kube-api-access-sshk4" (OuterVolumeSpecName: "kube-api-access-sshk4") pod "d8fd1789-891d-44f6-b16a-71ab32ec4ae0" (UID: "d8fd1789-891d-44f6-b16a-71ab32ec4ae0"). InnerVolumeSpecName "kube-api-access-sshk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:48:37.499284 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.499203    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:48:37.499284 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.499231    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh7sf\" (UniqueName: \"kubernetes.io/projected/12153e69-72a7-4bc2-b7b4-08744d8b5c2a-kube-api-access-xh7sf\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:48:37.499284 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.499242    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sshk4\" (UniqueName: \"kubernetes.io/projected/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-kube-api-access-sshk4\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:48:37.499284 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.499250    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8fd1789-891d-44f6-b16a-71ab32ec4ae0-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:48:37.630038 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:37.630004    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:38.102917 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.102884    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74cfcfce-882f-46be-bce9-750fe421f0a3" path="/var/lib/kubelet/pods/74cfcfce-882f-46be-bce9-750fe421f0a3/volumes"
Sep 09 12:48:38.115915 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.115894    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"
Sep 09 12:48:38.115915 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.115901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg" event={"ID":"12153e69-72a7-4bc2-b7b4-08744d8b5c2a","Type":"ContainerDied","Data":"a094a916b8e38264bf301c953a519ddc444e67178e6dac2ddabb7bb37cc8cd62"}
Sep 09 12:48:38.116062 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.115961    2579 scope.go:117] "RemoveContainer" containerID="7d879971bf22b023d2b2c0ae4a090b6d31f4627f128ddf8c19a90faaa996d269"
Sep 09 12:48:38.117388 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.117365    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"
Sep 09 12:48:38.117500 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.117408    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl" event={"ID":"d8fd1789-891d-44f6-b16a-71ab32ec4ae0","Type":"ContainerDied","Data":"7616ff1e03e609c6f056e98120430ee076d864666c631446ab72e3b9b2ad8228"}
Sep 09 12:48:38.125322 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.125303    2579 scope.go:117] "RemoveContainer" containerID="e93c5096fd879842078345bab3594c2c0533f99ce8e5f4297faedf6e3310a7b0"
Sep 09 12:48:38.134522 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.134502    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"]
Sep 09 12:48:38.140021 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.139995    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-610d01-564f88b77-5pwnl"]
Sep 09 12:48:38.149663 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.149628    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"]
Sep 09 12:48:38.155626 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:38.155603    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-610d01-b85b48fb6-v22cg"]
Sep 09 12:48:40.102175 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:40.102140    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12153e69-72a7-4bc2-b7b4-08744d8b5c2a" path="/var/lib/kubelet/pods/12153e69-72a7-4bc2-b7b4-08744d8b5c2a/volumes"
Sep 09 12:48:40.102562 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:40.102515    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8fd1789-891d-44f6-b16a-71ab32ec4ae0" path="/var/lib/kubelet/pods/d8fd1789-891d-44f6-b16a-71ab32ec4ae0/volumes"
Sep 09 12:48:42.550274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:48:42.550240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:02.657211 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:02.657160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:13.196809 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:13.196775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:13.258094 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:13.258054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:19.187169 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:19.187134    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:20.084730 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:20.084695    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:26.027518 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.027482    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:26.157929 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.157889    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:26.328447 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.328359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:26.663839 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.663760    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"]
Sep 09 12:49:26.664082 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664070    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74cfcfce-882f-46be-bce9-750fe421f0a3" containerName="revoke-keys"
Sep 09 12:49:26.664124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664083    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="74cfcfce-882f-46be-bce9-750fe421f0a3" containerName="revoke-keys"
Sep 09 12:49:26.664124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664098    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8fd1789-891d-44f6-b16a-71ab32ec4ae0" containerName="payload-processing"
Sep 09 12:49:26.664124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664103    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8fd1789-891d-44f6-b16a-71ab32ec4ae0" containerName="payload-processing"
Sep 09 12:49:26.664124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664120    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12153e69-72a7-4bc2-b7b4-08744d8b5c2a" containerName="payload-pre-processing"
Sep 09 12:49:26.664245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664126    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="12153e69-72a7-4bc2-b7b4-08744d8b5c2a" containerName="payload-pre-processing"
Sep 09 12:49:26.664245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664189    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="74cfcfce-882f-46be-bce9-750fe421f0a3" containerName="revoke-keys"
Sep 09 12:49:26.664245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664197    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8fd1789-891d-44f6-b16a-71ab32ec4ae0" containerName="payload-processing"
Sep 09 12:49:26.664245 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.664201    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="12153e69-72a7-4bc2-b7b4-08744d8b5c2a" containerName="payload-pre-processing"
Sep 09 12:49:26.666882 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.666867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"
Sep 09 12:49:26.668881 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.668856    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:49:26.669021 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.668857    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:49:26.674871 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.674845    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"]
Sep 09 12:49:26.817388 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.817348    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddecc17b-dbf8-47fe-a7bc-e9522949045d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf\" (UID: \"ddecc17b-dbf8-47fe-a7bc-e9522949045d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"
Sep 09 12:49:26.868279 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.868240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:26.918484 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.918409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddecc17b-dbf8-47fe-a7bc-e9522949045d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf\" (UID: \"ddecc17b-dbf8-47fe-a7bc-e9522949045d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"
Sep 09 12:49:26.919048 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.919029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddecc17b-dbf8-47fe-a7bc-e9522949045d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf\" (UID: \"ddecc17b-dbf8-47fe-a7bc-e9522949045d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"
Sep 09 12:49:26.978487 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:26.978452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"
Sep 09 12:49:27.247267 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:27.247239    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"]
Sep 09 12:49:27.249132 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:49:27.249102    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddecc17b_dbf8_47fe_a7bc_e9522949045d.slice/crio-0e8b937334e8e1c3cb10c27a4e9cea104af58f2584e54ff6cac8c044cf81e712 WatchSource:0}: Error finding container 0e8b937334e8e1c3cb10c27a4e9cea104af58f2584e54ff6cac8c044cf81e712: Status 404 returned error can't find the container with id 0e8b937334e8e1c3cb10c27a4e9cea104af58f2584e54ff6cac8c044cf81e712
Sep 09 12:49:27.353138 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:27.353101    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf" event={"ID":"ddecc17b-dbf8-47fe-a7bc-e9522949045d","Type":"ContainerStarted","Data":"0e8b937334e8e1c3cb10c27a4e9cea104af58f2584e54ff6cac8c044cf81e712"}
Sep 09 12:49:28.359620 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:28.359579    2579 generic.go:356] "Generic (PLEG): container finished" podID="ddecc17b-dbf8-47fe-a7bc-e9522949045d" containerID="13d22fc25d2bb6cfe2acfa4700a7e527b0d1bfdfbcf8f81c8383de34e8eaaf80" exitCode=0
Sep 09 12:49:28.360018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:28.359660    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf" event={"ID":"ddecc17b-dbf8-47fe-a7bc-e9522949045d","Type":"ContainerDied","Data":"13d22fc25d2bb6cfe2acfa4700a7e527b0d1bfdfbcf8f81c8383de34e8eaaf80"}
Sep 09 12:49:29.494095 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:29.494070    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"
Sep 09 12:49:29.643871 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:29.643769    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddecc17b-dbf8-47fe-a7bc-e9522949045d-maas-api-service-ca\") pod \"ddecc17b-dbf8-47fe-a7bc-e9522949045d\" (UID: \"ddecc17b-dbf8-47fe-a7bc-e9522949045d\") "
Sep 09 12:49:29.644135 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:29.644112    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddecc17b-dbf8-47fe-a7bc-e9522949045d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ddecc17b-dbf8-47fe-a7bc-e9522949045d" (UID: "ddecc17b-dbf8-47fe-a7bc-e9522949045d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:29.745311 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:29.745271    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddecc17b-dbf8-47fe-a7bc-e9522949045d-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:30.323201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:30.323162    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:30.373999 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:30.373918    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"
Sep 09 12:49:30.373999 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:30.373948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf" event={"ID":"ddecc17b-dbf8-47fe-a7bc-e9522949045d","Type":"ContainerDied","Data":"0e8b937334e8e1c3cb10c27a4e9cea104af58f2584e54ff6cac8c044cf81e712"}
Sep 09 12:49:30.374179 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:30.374005    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e8b937334e8e1c3cb10c27a4e9cea104af58f2584e54ff6cac8c044cf81e712"
Sep 09 12:49:32.004158 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:32.004119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:34.720205 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:34.720169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:37.063320 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.063235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:37.653999 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.653965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:37.678392 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.678349    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"]
Sep 09 12:49:37.678653 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.678608    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t" podUID="9eaa666b-80b4-4c94-978b-7dccca53990f" containerName="payload-processing" containerID="cri-o://1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709" gracePeriod=30
Sep 09 12:49:37.754018 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.753977    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:37.845345 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.845283    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"]
Sep 09 12:49:37.847325 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.847290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:37.849261 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.849235    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2db0a6-rkkcf"]
Sep 09 12:49:37.934400 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.934178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:37.934722 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:37.934616    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:49:38.102810 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.102776    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddecc17b-dbf8-47fe-a7bc-e9522949045d" path="/var/lib/kubelet/pods/ddecc17b-dbf8-47fe-a7bc-e9522949045d/volumes"
Sep 09 12:49:38.120828 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.120793    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eaa666b-80b4-4c94-978b-7dccca53990f-plugins-config-volume\") pod \"9eaa666b-80b4-4c94-978b-7dccca53990f\" (UID: \"9eaa666b-80b4-4c94-978b-7dccca53990f\") "
Sep 09 12:49:38.120974 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.120870    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4jfs\" (UniqueName: \"kubernetes.io/projected/9eaa666b-80b4-4c94-978b-7dccca53990f-kube-api-access-l4jfs\") pod \"9eaa666b-80b4-4c94-978b-7dccca53990f\" (UID: \"9eaa666b-80b4-4c94-978b-7dccca53990f\") "
Sep 09 12:49:38.121156 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.121131    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9eaa666b-80b4-4c94-978b-7dccca53990f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9eaa666b-80b4-4c94-978b-7dccca53990f" (UID: "9eaa666b-80b4-4c94-978b-7dccca53990f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:38.122941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.122921    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eaa666b-80b4-4c94-978b-7dccca53990f-kube-api-access-l4jfs" (OuterVolumeSpecName: "kube-api-access-l4jfs") pod "9eaa666b-80b4-4c94-978b-7dccca53990f" (UID: "9eaa666b-80b4-4c94-978b-7dccca53990f"). InnerVolumeSpecName "kube-api-access-l4jfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:38.222376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.222337    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eaa666b-80b4-4c94-978b-7dccca53990f-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:38.222376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.222368    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4jfs\" (UniqueName: \"kubernetes.io/projected/9eaa666b-80b4-4c94-978b-7dccca53990f-kube-api-access-l4jfs\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:38.247230 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.247194    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:38.415023 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.414988    2579 generic.go:356] "Generic (PLEG): container finished" podID="9eaa666b-80b4-4c94-978b-7dccca53990f" containerID="1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709" exitCode=0
Sep 09 12:49:38.415197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.415039    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"
Sep 09 12:49:38.415197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.415062    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t" event={"ID":"9eaa666b-80b4-4c94-978b-7dccca53990f","Type":"ContainerDied","Data":"1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709"}
Sep 09 12:49:38.415197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.415102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t" event={"ID":"9eaa666b-80b4-4c94-978b-7dccca53990f","Type":"ContainerDied","Data":"a89a343a91f2d88cf8c76bcaed652a8f2774fcf0eac6d92f63f6cdafe1eb5705"}
Sep 09 12:49:38.415197 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.415124    2579 scope.go:117] "RemoveContainer" containerID="1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709"
Sep 09 12:49:38.423919 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.423902    2579 scope.go:117] "RemoveContainer" containerID="1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709"
Sep 09 12:49:38.424170 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:49:38.424153    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709\": container with ID starting with 1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709 not found: ID does not exist" containerID="1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709"
Sep 09 12:49:38.424242 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.424176    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709"} err="failed to get container status \"1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709\": rpc error: code = NotFound desc = could not find container \"1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709\": container with ID starting with 1567fcc41b27c799210cde230169ad72638a5429f33dee12e1071f723065c709 not found: ID does not exist"
Sep 09 12:49:38.435614 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.435587    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"]
Sep 09 12:49:38.439755 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:38.439732    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2db0a6-678b564949-7rx6t"]
Sep 09 12:49:40.103282 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:40.103249    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9eaa666b-80b4-4c94-978b-7dccca53990f" path="/var/lib/kubelet/pods/9eaa666b-80b4-4c94-978b-7dccca53990f/volumes"
Sep 09 12:49:44.367704 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.367666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:44.467086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.467051    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:44.567191 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.567145    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"]
Sep 09 12:49:44.567411 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.567366    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy" containerID="cri-o://9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893" gracePeriod=30
Sep 09 12:49:44.614618 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.614584    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:44.942021 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.941978    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"]
Sep 09 12:49:44.942394 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.942376    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddecc17b-dbf8-47fe-a7bc-e9522949045d" containerName="revoke-keys"
Sep 09 12:49:44.942484 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.942396    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddecc17b-dbf8-47fe-a7bc-e9522949045d" containerName="revoke-keys"
Sep 09 12:49:44.942484 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.942440    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eaa666b-80b4-4c94-978b-7dccca53990f" containerName="payload-processing"
Sep 09 12:49:44.942484 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.942449    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eaa666b-80b4-4c94-978b-7dccca53990f" containerName="payload-processing"
Sep 09 12:49:44.942655 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.942530    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eaa666b-80b4-4c94-978b-7dccca53990f" containerName="payload-processing"
Sep 09 12:49:44.942655 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.942545    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddecc17b-dbf8-47fe-a7bc-e9522949045d" containerName="revoke-keys"
Sep 09 12:49:44.946748 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.946726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"
Sep 09 12:49:44.948820 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.948799    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:49:44.948820 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.948814    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:49:44.953184 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.953159    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"]
Sep 09 12:49:44.977493 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:44.977462    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfa5755c-8c3c-4073-b195-02a0e4d2c0ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9\" (UID: \"bfa5755c-8c3c-4073-b195-02a0e4d2c0ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"
Sep 09 12:49:45.078123 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:45.078087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfa5755c-8c3c-4073-b195-02a0e4d2c0ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9\" (UID: \"bfa5755c-8c3c-4073-b195-02a0e4d2c0ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"
Sep 09 12:49:45.078742 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:45.078725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfa5755c-8c3c-4073-b195-02a0e4d2c0ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9\" (UID: \"bfa5755c-8c3c-4073-b195-02a0e4d2c0ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"
Sep 09 12:49:45.147862 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:45.147826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:45.259543 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:45.259512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"
Sep 09 12:49:45.531503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:45.531473    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"]
Sep 09 12:49:45.533234 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:49:45.533210    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa5755c_8c3c_4073_b195_02a0e4d2c0ed.slice/crio-03fa08e0bc63a643f25f1b3963dd228cc39104c8a5480451a3df531c9626e70c WatchSource:0}: Error finding container 03fa08e0bc63a643f25f1b3963dd228cc39104c8a5480451a3df531c9626e70c: Status 404 returned error can't find the container with id 03fa08e0bc63a643f25f1b3963dd228cc39104c8a5480451a3df531c9626e70c
Sep 09 12:49:46.457057 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:46.457028    2579 generic.go:356] "Generic (PLEG): container finished" podID="bfa5755c-8c3c-4073-b195-02a0e4d2c0ed" containerID="03bf9237747212cac36932332f8b4e3fc9212acff55371c1c4e6bdff7961a5c0" exitCode=0
Sep 09 12:49:46.457215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:46.457116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9" event={"ID":"bfa5755c-8c3c-4073-b195-02a0e4d2c0ed","Type":"ContainerDied","Data":"03bf9237747212cac36932332f8b4e3fc9212acff55371c1c4e6bdff7961a5c0"}
Sep 09 12:49:46.457215 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:46.457176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9" event={"ID":"bfa5755c-8c3c-4073-b195-02a0e4d2c0ed","Type":"ContainerStarted","Data":"03fa08e0bc63a643f25f1b3963dd228cc39104c8a5480451a3df531c9626e70c"}
Sep 09 12:49:47.587500 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:47.587474    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"
Sep 09 12:49:47.600373 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:47.600354    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfa5755c-8c3c-4073-b195-02a0e4d2c0ed-maas-api-service-ca\") pod \"bfa5755c-8c3c-4073-b195-02a0e4d2c0ed\" (UID: \"bfa5755c-8c3c-4073-b195-02a0e4d2c0ed\") "
Sep 09 12:49:47.600690 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:47.600669    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfa5755c-8c3c-4073-b195-02a0e4d2c0ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bfa5755c-8c3c-4073-b195-02a0e4d2c0ed" (UID: "bfa5755c-8c3c-4073-b195-02a0e4d2c0ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:47.701232 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:47.701200    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfa5755c-8c3c-4073-b195-02a0e4d2c0ed-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:48.468503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:48.468469    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"
Sep 09 12:49:48.468503 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:48.468477    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9" event={"ID":"bfa5755c-8c3c-4073-b195-02a0e4d2c0ed","Type":"ContainerDied","Data":"03fa08e0bc63a643f25f1b3963dd228cc39104c8a5480451a3df531c9626e70c"}
Sep 09 12:49:48.468736 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:48.468526    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03fa08e0bc63a643f25f1b3963dd228cc39104c8a5480451a3df531c9626e70c"
Sep 09 12:49:49.820208 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.820181    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:49:49.920919 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.920823    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-credential-socket\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.920919 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.920869    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-certs\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.920919 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.920887    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-envoy\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.921201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.920935    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-data\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.921201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.920964    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-istio-token\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.921201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.920997    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4743e947-fe49-4b95-9346-92879a434e40-istiod-ca-cert\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.921201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921020    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-socket\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.921201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921113    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:49.921201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921148    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrmpw\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-kube-api-access-qrmpw\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.921201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921192    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4743e947-fe49-4b95-9346-92879a434e40-istio-podinfo\") pod \"4743e947-fe49-4b95-9346-92879a434e40\" (UID: \"4743e947-fe49-4b95-9346-92879a434e40\") "
Sep 09 12:49:49.921201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921194    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:49.921616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921294    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:49.921616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921309    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-data" (OuterVolumeSpecName: "istio-data") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:49.921616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921434    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4743e947-fe49-4b95-9346-92879a434e40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:49.921616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921468    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-credential-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:49.921616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921492    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-certs\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:49.921616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921509    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-data\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:49.921616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.921523    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-workload-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:49.923323 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.923299    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4743e947-fe49-4b95-9346-92879a434e40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:49:49.923427 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.923359    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-istio-token" (OuterVolumeSpecName: "istio-token") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:49.923427 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.923359    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-kube-api-access-qrmpw" (OuterVolumeSpecName: "kube-api-access-qrmpw") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "kube-api-access-qrmpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:49.923427 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:49.923390    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4743e947-fe49-4b95-9346-92879a434e40" (UID: "4743e947-fe49-4b95-9346-92879a434e40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:50.022944 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.022911    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4743e947-fe49-4b95-9346-92879a434e40-istio-envoy\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:50.022944 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.022938    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-istio-token\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:50.022944 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.022948    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4743e947-fe49-4b95-9346-92879a434e40-istiod-ca-cert\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:50.023175 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.022957    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrmpw\" (UniqueName: \"kubernetes.io/projected/4743e947-fe49-4b95-9346-92879a434e40-kube-api-access-qrmpw\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:50.023175 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.022965    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4743e947-fe49-4b95-9346-92879a434e40-istio-podinfo\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:50.480092 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.480060    2579 generic.go:356] "Generic (PLEG): container finished" podID="4743e947-fe49-4b95-9346-92879a434e40" containerID="9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893" exitCode=0
Sep 09 12:49:50.480274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.480103    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" event={"ID":"4743e947-fe49-4b95-9346-92879a434e40","Type":"ContainerDied","Data":"9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893"}
Sep 09 12:49:50.480274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.480110    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"
Sep 09 12:49:50.480274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.480145    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59" event={"ID":"4743e947-fe49-4b95-9346-92879a434e40","Type":"ContainerDied","Data":"e110a45d23b5b214f20666c8ca7e7614014e0fe7e83301e7590bc34a4003c1c3"}
Sep 09 12:49:50.480274 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.480167    2579 scope.go:117] "RemoveContainer" containerID="9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893"
Sep 09 12:49:50.488918 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.488898    2579 scope.go:117] "RemoveContainer" containerID="185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49"
Sep 09 12:49:50.496777 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.496756    2579 scope.go:117] "RemoveContainer" containerID="9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893"
Sep 09 12:49:50.497073 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:49:50.497048    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893\": container with ID starting with 9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893 not found: ID does not exist" containerID="9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893"
Sep 09 12:49:50.497179 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.497084    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893"} err="failed to get container status \"9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893\": rpc error: code = NotFound desc = could not find container \"9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893\": container with ID starting with 9d40347cabe169bb852f3a8699919015d277cfbc9edf1aa9fe4c44be46529893 not found: ID does not exist"
Sep 09 12:49:50.497179 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.497106    2579 scope.go:117] "RemoveContainer" containerID="185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49"
Sep 09 12:49:50.497406 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:49:50.497387    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49\": container with ID starting with 185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49 not found: ID does not exist" containerID="185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49"
Sep 09 12:49:50.497449 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.497413    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49"} err="failed to get container status \"185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49\": rpc error: code = NotFound desc = could not find container \"185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49\": container with ID starting with 185fd678fdc31b4eff4dd6400572db421730e15909e21265a69287ce0159cd49 not found: ID does not exist"
Sep 09 12:49:50.497788 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.497771    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"]
Sep 09 12:49:50.503717 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:50.503695    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-2db0a6-openshift-default-55c4b657c8-sbx59"]
Sep 09 12:49:52.102553 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:52.102498    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4743e947-fe49-4b95-9346-92879a434e40" path="/var/lib/kubelet/pods/4743e947-fe49-4b95-9346-92879a434e40/volumes"
Sep 09 12:49:55.494862 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.494823    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"]
Sep 09 12:49:55.495282 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.494954    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:55.502993 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.502961    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-283adb-29815965-9kg2t"]
Sep 09 12:49:55.530221 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.530188    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"]
Sep 09 12:49:55.530461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.530431    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2" podUID="eb19cc84-83dd-45ed-94c0-8fb4edb69ec5" containerName="payload-pre-processing" containerID="cri-o://2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167" gracePeriod=30
Sep 09 12:49:55.594446 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.594405    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:55.698195 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.698156    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:55.740243 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.740210    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"]
Sep 09 12:49:55.743059 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.743031    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-283adb-h7ms9"]
Sep 09 12:49:55.778113 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.778079    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:49:55.796610 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.796572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:55.869432 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.869397    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4p76\" (UniqueName: \"kubernetes.io/projected/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-kube-api-access-v4p76\") pod \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\" (UID: \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\") "
Sep 09 12:49:55.869604 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.869462    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-plugins-config-volume\") pod \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\" (UID: \"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5\") "
Sep 09 12:49:55.869933 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.869901    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb19cc84-83dd-45ed-94c0-8fb4edb69ec5" (UID: "eb19cc84-83dd-45ed-94c0-8fb4edb69ec5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:55.871591 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.871567    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-kube-api-access-v4p76" (OuterVolumeSpecName: "kube-api-access-v4p76") pod "eb19cc84-83dd-45ed-94c0-8fb4edb69ec5" (UID: "eb19cc84-83dd-45ed-94c0-8fb4edb69ec5"). InnerVolumeSpecName "kube-api-access-v4p76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:55.970331 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.970287    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4p76\" (UniqueName: \"kubernetes.io/projected/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-kube-api-access-v4p76\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:55.970331 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:55.970322    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:49:56.102901 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.102817    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fb78d50-753f-483e-8f43-6ef6f6d78cf9" path="/var/lib/kubelet/pods/7fb78d50-753f-483e-8f43-6ef6f6d78cf9/volumes"
Sep 09 12:49:56.103123 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.103110    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfa5755c-8c3c-4073-b195-02a0e4d2c0ed" path="/var/lib/kubelet/pods/bfa5755c-8c3c-4073-b195-02a0e4d2c0ed/volumes"
Sep 09 12:49:56.123045 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.123016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:56.517493 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.517458    2579 generic.go:356] "Generic (PLEG): container finished" podID="eb19cc84-83dd-45ed-94c0-8fb4edb69ec5" containerID="2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167" exitCode=0
Sep 09 12:49:56.517977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.517514    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"
Sep 09 12:49:56.517977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.517543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2" event={"ID":"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5","Type":"ContainerDied","Data":"2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167"}
Sep 09 12:49:56.517977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.517595    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2" event={"ID":"eb19cc84-83dd-45ed-94c0-8fb4edb69ec5","Type":"ContainerDied","Data":"f7a9998fc9c37b288f348737e5d330706ab7a449a79975756e318f44be7fda3d"}
Sep 09 12:49:56.517977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.517619    2579 scope.go:117] "RemoveContainer" containerID="2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167"
Sep 09 12:49:56.525888 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.525866    2579 scope.go:117] "RemoveContainer" containerID="2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167"
Sep 09 12:49:56.526206 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:49:56.526179    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167\": container with ID starting with 2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167 not found: ID does not exist" containerID="2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167"
Sep 09 12:49:56.526262 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.526215    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167"} err="failed to get container status \"2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167\": rpc error: code = NotFound desc = could not find container \"2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167\": container with ID starting with 2bdaf8ffd7374de71139954196ae48d5759c140ed12cf3d3afef651eaadd8167 not found: ID does not exist"
Sep 09 12:49:56.532950 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.532923    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"]
Sep 09 12:49:56.535171 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:56.535139    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-283adb-65598dbf8-vvvh2"]
Sep 09 12:49:58.102108 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:49:58.102069    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb19cc84-83dd-45ed-94c0-8fb4edb69ec5" path="/var/lib/kubelet/pods/eb19cc84-83dd-45ed-94c0-8fb4edb69ec5/volumes"
Sep 09 12:50:02.549324 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:02.549280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:02.649779 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:02.649741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:02.797796 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:02.797759    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"]
Sep 09 12:50:02.798025 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:02.797986    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy" containerID="cri-o://88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2" gracePeriod=30
Sep 09 12:50:02.822951 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:02.822873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:03.148969 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.148871    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"]
Sep 09 12:50:03.149206 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149194    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb19cc84-83dd-45ed-94c0-8fb4edb69ec5" containerName="payload-pre-processing"
Sep 09 12:50:03.149247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149207    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb19cc84-83dd-45ed-94c0-8fb4edb69ec5" containerName="payload-pre-processing"
Sep 09 12:50:03.149247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149223    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy"
Sep 09 12:50:03.149247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149229    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy"
Sep 09 12:50:03.149247 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149243    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfa5755c-8c3c-4073-b195-02a0e4d2c0ed" containerName="revoke-keys"
Sep 09 12:50:03.149376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149248    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfa5755c-8c3c-4073-b195-02a0e4d2c0ed" containerName="revoke-keys"
Sep 09 12:50:03.149376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149300    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy"
Sep 09 12:50:03.149376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149309    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb19cc84-83dd-45ed-94c0-8fb4edb69ec5" containerName="payload-pre-processing"
Sep 09 12:50:03.149376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149315    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy"
Sep 09 12:50:03.149376 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.149322    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfa5755c-8c3c-4073-b195-02a0e4d2c0ed" containerName="revoke-keys"
Sep 09 12:50:03.153678 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.153628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"
Sep 09 12:50:03.155853 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.155826    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:50:03.155986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.155902    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wvr9\""
Sep 09 12:50:03.158746 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.158717    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"]
Sep 09 12:50:03.235926 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.235881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/007eff4a-0ed2-48bf-b690-752f0d5c8101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx\" (UID: \"007eff4a-0ed2-48bf-b690-752f0d5c8101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"
Sep 09 12:50:03.336575 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.336533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/007eff4a-0ed2-48bf-b690-752f0d5c8101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx\" (UID: \"007eff4a-0ed2-48bf-b690-752f0d5c8101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"
Sep 09 12:50:03.337178 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.337158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/007eff4a-0ed2-48bf-b690-752f0d5c8101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx\" (UID: \"007eff4a-0ed2-48bf-b690-752f0d5c8101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"
Sep 09 12:50:03.466467 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.466420    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"
Sep 09 12:50:03.495609 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.495571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:03.735383 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:03.735353    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"]
Sep 09 12:50:03.737359 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:50:03.737332    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007eff4a_0ed2_48bf_b690_752f0d5c8101.slice/crio-a3110d405c38c387817f7de7f068267ec67db4205e66a5c3c7a30d90ef53337b WatchSource:0}: Error finding container a3110d405c38c387817f7de7f068267ec67db4205e66a5c3c7a30d90ef53337b: Status 404 returned error can't find the container with id a3110d405c38c387817f7de7f068267ec67db4205e66a5c3c7a30d90ef53337b
Sep 09 12:50:04.568941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:04.568904    2579 generic.go:356] "Generic (PLEG): container finished" podID="007eff4a-0ed2-48bf-b690-752f0d5c8101" containerID="04503d7587b0945a202db16446899a04e5623183d50db1f56432a9923f7d67ac" exitCode=0
Sep 09 12:50:04.569110 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:04.568992    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx" event={"ID":"007eff4a-0ed2-48bf-b690-752f0d5c8101","Type":"ContainerDied","Data":"04503d7587b0945a202db16446899a04e5623183d50db1f56432a9923f7d67ac"}
Sep 09 12:50:04.569110 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:04.569054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx" event={"ID":"007eff4a-0ed2-48bf-b690-752f0d5c8101","Type":"ContainerStarted","Data":"a3110d405c38c387817f7de7f068267ec67db4205e66a5c3c7a30d90ef53337b"}
Sep 09 12:50:05.695443 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:05.695415    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"
Sep 09 12:50:05.753898 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:05.753865    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/007eff4a-0ed2-48bf-b690-752f0d5c8101-maas-api-service-ca\") pod \"007eff4a-0ed2-48bf-b690-752f0d5c8101\" (UID: \"007eff4a-0ed2-48bf-b690-752f0d5c8101\") "
Sep 09 12:50:05.754249 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:05.754228    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/007eff4a-0ed2-48bf-b690-752f0d5c8101-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "007eff4a-0ed2-48bf-b690-752f0d5c8101" (UID: "007eff4a-0ed2-48bf-b690-752f0d5c8101"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:05.854490 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:05.854405    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/007eff4a-0ed2-48bf-b690-752f0d5c8101-maas-api-service-ca\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:06.579866 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:06.579837    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"
Sep 09 12:50:06.580045 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:06.579875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx" event={"ID":"007eff4a-0ed2-48bf-b690-752f0d5c8101","Type":"ContainerDied","Data":"a3110d405c38c387817f7de7f068267ec67db4205e66a5c3c7a30d90ef53337b"}
Sep 09 12:50:06.580045 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:06.579923    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3110d405c38c387817f7de7f068267ec67db4205e66a5c3c7a30d90ef53337b"
Sep 09 12:50:08.046708 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.046681    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:50:08.173872 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.173779    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-istio-token\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.173872 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.173828    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-data\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.173872 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.173867    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prtc7\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-kube-api-access-prtc7\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.174174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.173888    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1de361f-3185-4696-8908-0bcf54baf582-istio-podinfo\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.174174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.173943    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-certs\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.174174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.173968    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-envoy\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.174174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174009    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-socket\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.174174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174034    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-credential-socket\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.174174 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174063    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1de361f-3185-4696-8908-0bcf54baf582-istiod-ca-cert\") pod \"c1de361f-3185-4696-8908-0bcf54baf582\" (UID: \"c1de361f-3185-4696-8908-0bcf54baf582\") "
Sep 09 12:50:08.174514 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174239    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-data" (OuterVolumeSpecName: "istio-data") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:08.174514 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174379    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:08.174514 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174502    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-data\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.174693 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174525    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.174693 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174563    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:08.174693 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174661    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1de361f-3185-4696-8908-0bcf54baf582-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:08.174693 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.174671    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:08.176239 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.176207    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-istio-token" (OuterVolumeSpecName: "istio-token") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:08.176239 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.176207    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1de361f-3185-4696-8908-0bcf54baf582-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 12:50:08.176478 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.176411    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-kube-api-access-prtc7" (OuterVolumeSpecName: "kube-api-access-prtc7") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "kube-api-access-prtc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:08.176548 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.176532    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c1de361f-3185-4696-8908-0bcf54baf582" (UID: "c1de361f-3185-4696-8908-0bcf54baf582"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:08.275770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.275732    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-workload-certs\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.275770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.275764    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-istio-envoy\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.275770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.275772    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1de361f-3185-4696-8908-0bcf54baf582-credential-socket\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.275770 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.275781    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1de361f-3185-4696-8908-0bcf54baf582-istiod-ca-cert\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.276051 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.275791    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-istio-token\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.276051 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.275798    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prtc7\" (UniqueName: \"kubernetes.io/projected/c1de361f-3185-4696-8908-0bcf54baf582-kube-api-access-prtc7\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.276051 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.275808    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1de361f-3185-4696-8908-0bcf54baf582-istio-podinfo\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:08.596765 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.596730    2579 generic.go:356] "Generic (PLEG): container finished" podID="c1de361f-3185-4696-8908-0bcf54baf582" containerID="88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2" exitCode=0
Sep 09 12:50:08.596941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.596782    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"
Sep 09 12:50:08.596941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.596813    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" event={"ID":"c1de361f-3185-4696-8908-0bcf54baf582","Type":"ContainerDied","Data":"88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2"}
Sep 09 12:50:08.596941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.596872    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp" event={"ID":"c1de361f-3185-4696-8908-0bcf54baf582","Type":"ContainerDied","Data":"691397cce8c773cdf6d14c965a7954e72ff8a09d905f982f1ec5fdb518434dab"}
Sep 09 12:50:08.596941 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.596894    2579 scope.go:117] "RemoveContainer" containerID="88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2"
Sep 09 12:50:08.605901 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.605884    2579 scope.go:117] "RemoveContainer" containerID="0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d"
Sep 09 12:50:08.616228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.616186    2579 scope.go:117] "RemoveContainer" containerID="88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2"
Sep 09 12:50:08.616553 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:50:08.616531    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2\": container with ID starting with 88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2 not found: ID does not exist" containerID="88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2"
Sep 09 12:50:08.616609 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.616560    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2"} err="failed to get container status \"88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2\": rpc error: code = NotFound desc = could not find container \"88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2\": container with ID starting with 88e424beabaac3b49ee63a728e564690cdc3b68f2a83da3a8559ece6b5a0bcd2 not found: ID does not exist"
Sep 09 12:50:08.616609 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.616578    2579 scope.go:117] "RemoveContainer" containerID="0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d"
Sep 09 12:50:08.616882 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:50:08.616862    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d\": container with ID starting with 0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d not found: ID does not exist" containerID="0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d"
Sep 09 12:50:08.616922 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.616891    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d"} err="failed to get container status \"0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d\": rpc error: code = NotFound desc = could not find container \"0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d\": container with ID starting with 0b4dbe40d260f40a1078f1bbeaa754565543558ef5713f8c903bc5b8f7b5d27d not found: ID does not exist"
Sep 09 12:50:08.617926 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.617909    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"]
Sep 09 12:50:08.621584 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:08.621563    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-283adb-openshift-default-969d8f876-fbbmp"]
Sep 09 12:50:10.102344 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:10.102312    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1de361f-3185-4696-8908-0bcf54baf582" path="/var/lib/kubelet/pods/c1de361f-3185-4696-8908-0bcf54baf582/volumes"
Sep 09 12:50:13.761524 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:13.761490    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"]
Sep 09 12:50:13.763894 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:13.763868    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-fe6606-29815965-x672v"]
Sep 09 12:50:13.767308 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:13.767278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:13.797390 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:13.797352    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"]
Sep 09 12:50:13.797649 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:13.797611    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8" podUID="fff11232-42ca-4288-a4dd-b99b124b144c" containerName="payload-pre-processing" containerID="cri-o://887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea" gracePeriod=30
Sep 09 12:50:13.848721 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:13.848683    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:13.958354 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:13.958318    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"]
Sep 09 12:50:13.961093 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:13.961063    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fe6606-fjmpx"]
Sep 09 12:50:14.065242 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.065217    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:50:14.102696 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.102656    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="007eff4a-0ed2-48bf-b690-752f0d5c8101" path="/var/lib/kubelet/pods/007eff4a-0ed2-48bf-b690-752f0d5c8101/volumes"
Sep 09 12:50:14.102957 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.102944    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e7a1e8b-d631-4c1f-896a-adad71bef3fa" path="/var/lib/kubelet/pods/7e7a1e8b-d631-4c1f-896a-adad71bef3fa/volumes"
Sep 09 12:50:14.129881 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.129838    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff11232-42ca-4288-a4dd-b99b124b144c-plugins-config-volume\") pod \"fff11232-42ca-4288-a4dd-b99b124b144c\" (UID: \"fff11232-42ca-4288-a4dd-b99b124b144c\") "
Sep 09 12:50:14.130064 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.129970    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9kkq\" (UniqueName: \"kubernetes.io/projected/fff11232-42ca-4288-a4dd-b99b124b144c-kube-api-access-x9kkq\") pod \"fff11232-42ca-4288-a4dd-b99b124b144c\" (UID: \"fff11232-42ca-4288-a4dd-b99b124b144c\") "
Sep 09 12:50:14.130228 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.130204    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fff11232-42ca-4288-a4dd-b99b124b144c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fff11232-42ca-4288-a4dd-b99b124b144c" (UID: "fff11232-42ca-4288-a4dd-b99b124b144c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:14.131958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.131942    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fff11232-42ca-4288-a4dd-b99b124b144c-kube-api-access-x9kkq" (OuterVolumeSpecName: "kube-api-access-x9kkq") pod "fff11232-42ca-4288-a4dd-b99b124b144c" (UID: "fff11232-42ca-4288-a4dd-b99b124b144c"). InnerVolumeSpecName "kube-api-access-x9kkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:14.230828 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.230789    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9kkq\" (UniqueName: \"kubernetes.io/projected/fff11232-42ca-4288-a4dd-b99b124b144c-kube-api-access-x9kkq\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:14.230828 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.230825    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fff11232-42ca-4288-a4dd-b99b124b144c-plugins-config-volume\") on node \"ip-10-0-140-93.ec2.internal\" DevicePath \"\""
Sep 09 12:50:14.411130 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.411032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:14.473116 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.473078    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:14.628574 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.628537    2579 generic.go:356] "Generic (PLEG): container finished" podID="fff11232-42ca-4288-a4dd-b99b124b144c" containerID="887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea" exitCode=0
Sep 09 12:50:14.628778 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.628580    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"
Sep 09 12:50:14.628778 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.628599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8" event={"ID":"fff11232-42ca-4288-a4dd-b99b124b144c","Type":"ContainerDied","Data":"887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea"}
Sep 09 12:50:14.628778 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.628654    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8" event={"ID":"fff11232-42ca-4288-a4dd-b99b124b144c","Type":"ContainerDied","Data":"00019a33811c117b86d00f51f587229fe3bd26b130edccde7459e25ead5ba558"}
Sep 09 12:50:14.628778 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.628675    2579 scope.go:117] "RemoveContainer" containerID="887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea"
Sep 09 12:50:14.637719 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.637698    2579 scope.go:117] "RemoveContainer" containerID="887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea"
Sep 09 12:50:14.638045 ip-10-0-140-93 kubenswrapper[2579]: E0909 12:50:14.638028    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea\": container with ID starting with 887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea not found: ID does not exist" containerID="887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea"
Sep 09 12:50:14.638106 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.638055    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea"} err="failed to get container status \"887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea\": rpc error: code = NotFound desc = could not find container \"887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea\": container with ID starting with 887322792b16b2c5a13f2cf8d1842b981cdd453bb2b096781cb1b16159d309ea not found: ID does not exist"
Sep 09 12:50:14.649001 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.648970    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"]
Sep 09 12:50:14.652786 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:14.652757    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fe6606-d64b8f86b-9qqx8"]
Sep 09 12:50:16.102201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:16.102161    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fff11232-42ca-4288-a4dd-b99b124b144c" path="/var/lib/kubelet/pods/fff11232-42ca-4288-a4dd-b99b124b144c/volumes"
Sep 09 12:50:20.763209 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:20.763173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:21.990078 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:21.990044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:44.436401 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:50:44.436367    2579 scope.go:117] "RemoveContainer" containerID="622b805ac7691066303d38521a9efe6ab4f522d2e983a4d7c60f1ad7da68048c"
Sep 09 12:51:44.471953 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:51:44.471905    2579 scope.go:117] "RemoveContainer" containerID="b37955e494472a6d97c5ca288c49af402f19da0bcf1f0c646fcacd89435d0a53"
Sep 09 12:51:44.479811 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:51:44.479777    2579 scope.go:117] "RemoveContainer" containerID="969b22b31d8c8a2425e4055d4eca5954731d107a370eb4c3913a1f089ee8ad4d"
Sep 09 12:51:44.486890 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:51:44.486869    2579 scope.go:117] "RemoveContainer" containerID="501f74662d8df0c5e10fc023b7f1cf5ec4871ab7efea1d6febcf758418e91d33"
Sep 09 12:51:44.493654 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:51:44.493619    2579 scope.go:117] "RemoveContainer" containerID="866044ff50db3893bd74d002a9784817557b33383ec0e1109f5cd85ed8aa88ff"
Sep 09 12:51:44.500317 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:51:44.500299    2579 scope.go:117] "RemoveContainer" containerID="cc9502990f06222e2085195268619e7a18a0fae0423a43be61d0e14110c03f00"
Sep 09 12:51:46.600086 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:51:46.600040    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:47.305249 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:51:47.305217    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:48.879129 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:51:48.879064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:53:05.879612 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:53:05.879558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:53:18.008977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:53:18.008938    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:53:51.818872 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:53:51.818839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:23.553442 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:54:23.553396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:34.882986 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:54:34.882949    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:43.451994 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:54:43.451957    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:44.544155 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:54:44.544126    2579 scope.go:117] "RemoveContainer" containerID="4ddce4a2f74a1f546ccf42606605ba4af191cb414b3055aa6f5ef58a033b624a"
Sep 09 12:54:44.551848 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:54:44.551825    2579 scope.go:117] "RemoveContainer" containerID="eb833d50ad5f5c69cdd009b4a1fd84aec2325cae877da77cd8b8d70be8eba22e"
Sep 09 12:54:44.989653 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:54:44.989597    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:21.946201 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:55:21.946154    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:23.624131 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:55:23.624099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:44.572040 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:55:44.572006    2579 scope.go:117] "RemoveContainer" containerID="13d22fc25d2bb6cfe2acfa4700a7e527b0d1bfdfbcf8f81c8383de34e8eaaf80"
Sep 09 12:56:19.046207 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:19.046173    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kxrbl_38b74994-df95-4436-8813-1beba187cf21/server/0.log"
Sep 09 12:56:19.176316 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:19.176285    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j25dc_e33f9672-6676-4ee2-a0c8-7d0185baeeb2/manager/0.log"
Sep 09 12:56:19.432600 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:19.432522    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5d49f968-vmmnf_e589d913-a2d5-42cb-beef-deb4194efcdc/copy-manifests/0.log"
Sep 09 12:56:19.439237 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:19.439212    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5d49f968-vmmnf_e589d913-a2d5-42cb-beef-deb4194efcdc/manager/0.log"
Sep 09 12:56:20.171589 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:20.171558    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29815965-5khpl_60a9e3de-3508-47db-bd8e-58691ce92c7c/cleanup/0.log"
Sep 09 12:56:22.175441 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:22.175410    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799czcjgf_8da005ca-91cf-4a00-a21b-7c1291a79f29/istio-proxy/0.log"
Sep 09 12:56:22.300007 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:22.299979    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-n9cd4_3f56633f-41e1-4539-af32-8462cf7576de/discovery/0.log"
Sep 09 12:56:22.786369 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:22.786333    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2zjb7_5c881f1f-f3ec-4752-834e-2a694ab52ce2/payload-pre-processing/0.log"
Sep 09 12:56:23.376720 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:23.376686    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-tzfqt_5aced2bc-6ec2-4fe6-96c5-78b60bb6ae88/main/0.log"
Sep 09 12:56:23.498672 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:23.498622    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vfw9j_7b41c861-0b06-4007-b75d-a5769b7f0140/main/0.log"
Sep 09 12:56:23.867298 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:23.867249    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68flrfpj_6565fd7e-8d72-4f3c-b457-46f0d86f76c0/main/0.log"
Sep 09 12:56:29.509993 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:29.509944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:29.895502 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:29.895382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:32.872536 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:32.872508    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d9ktm_47208fbf-65e4-4cfa-b2bd-7559528bdab4/global-pull-secret-syncer/0.log"
Sep 09 12:56:33.004150 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:33.004119    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log"
Sep 09 12:56:33.137298 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:33.137213    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-93.ec2.internal_f4f180e5ca224fd9a69355ef754b0028/haproxy/0.log"
Sep 09 12:56:37.514365 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:37.514299    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cz46c_f3a58883-1ecd-4f5b-8949-df19c7f1b4ce/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:40.118153 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:40.118113    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvh4h_4f82eed3-ba39-4e0f-a052-b40c22497062/node-exporter/0.log"
Sep 09 12:56:40.140072 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:40.140000    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvh4h_4f82eed3-ba39-4e0f-a052-b40c22497062/kube-rbac-proxy/0.log"
Sep 09 12:56:40.163333 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:40.163311    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vvh4h_4f82eed3-ba39-4e0f-a052-b40c22497062/init-textfile/0.log"
Sep 09 12:56:42.850292 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850253    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"]
Sep 09 12:56:42.850692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850588    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy"
Sep 09 12:56:42.850692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850602    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4743e947-fe49-4b95-9346-92879a434e40" containerName="istio-proxy"
Sep 09 12:56:42.850692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850612    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="007eff4a-0ed2-48bf-b690-752f0d5c8101" containerName="revoke-keys"
Sep 09 12:56:42.850692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850617    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="007eff4a-0ed2-48bf-b690-752f0d5c8101" containerName="revoke-keys"
Sep 09 12:56:42.850692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850626    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy"
Sep 09 12:56:42.850692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850630    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy"
Sep 09 12:56:42.850692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850666    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fff11232-42ca-4288-a4dd-b99b124b144c" containerName="payload-pre-processing"
Sep 09 12:56:42.850692 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850674    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fff11232-42ca-4288-a4dd-b99b124b144c" containerName="payload-pre-processing"
Sep 09 12:56:42.850964 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850727    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1de361f-3185-4696-8908-0bcf54baf582" containerName="istio-proxy"
Sep 09 12:56:42.850964 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850734    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="007eff4a-0ed2-48bf-b690-752f0d5c8101" containerName="revoke-keys"
Sep 09 12:56:42.850964 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.850742    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fff11232-42ca-4288-a4dd-b99b124b144c" containerName="payload-pre-processing"
Sep 09 12:56:42.853962 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.853942    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:42.856051 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.856033    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prpjx\"/\"kube-root-ca.crt\""
Sep 09 12:56:42.856616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.856599    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-prpjx\"/\"openshift-service-ca.crt\""
Sep 09 12:56:42.856700 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.856605    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-prpjx\"/\"default-dockercfg-lz68r\""
Sep 09 12:56:42.862009 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.861988    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"]
Sep 09 12:56:42.943167 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.943132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-podres\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:42.943313 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.943177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-sys\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:42.943313 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.943278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-lib-modules\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:42.943393 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.943334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-proc\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:42.943393 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:42.943359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28fgn\" (UniqueName: \"kubernetes.io/projected/db24e3b2-1a70-4092-b920-7874ae95957a-kube-api-access-28fgn\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.044616 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-proc\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.044785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28fgn\" (UniqueName: \"kubernetes.io/projected/db24e3b2-1a70-4092-b920-7874ae95957a-kube-api-access-28fgn\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.044785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-podres\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.044785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-sys\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.044785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-lib-modules\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.044785 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044739    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-proc\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.045002 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-podres\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.045002 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044822    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-sys\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.045002 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.044841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db24e3b2-1a70-4092-b920-7874ae95957a-lib-modules\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.052218 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.052193    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28fgn\" (UniqueName: \"kubernetes.io/projected/db24e3b2-1a70-4092-b920-7874ae95957a-kube-api-access-28fgn\") pod \"perf-node-gather-daemonset-pjbmv\" (UID: \"db24e3b2-1a70-4092-b920-7874ae95957a\") " pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.164550 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.164474    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:43.432283 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:43.432258    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"]
Sep 09 12:56:43.434359 ip-10-0-140-93 kubenswrapper[2579]: W0909 12:56:43.434315    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddb24e3b2_1a70_4092_b920_7874ae95957a.slice/crio-c1016e412d14aefe3193008d976e6fba7aa3ff17fc8e7b28bdac28d73717db49 WatchSource:0}: Error finding container c1016e412d14aefe3193008d976e6fba7aa3ff17fc8e7b28bdac28d73717db49: Status 404 returned error can't find the container with id c1016e412d14aefe3193008d976e6fba7aa3ff17fc8e7b28bdac28d73717db49
Sep 09 12:56:44.391208 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:44.391168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv" event={"ID":"db24e3b2-1a70-4092-b920-7874ae95957a","Type":"ContainerStarted","Data":"cb5f47c64267e77d1770931e94ba3380ba9799906c00aafba606a071d5638a13"}
Sep 09 12:56:44.391730 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:44.391234    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv" event={"ID":"db24e3b2-1a70-4092-b920-7874ae95957a","Type":"ContainerStarted","Data":"c1016e412d14aefe3193008d976e6fba7aa3ff17fc8e7b28bdac28d73717db49"}
Sep 09 12:56:44.391730 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:44.391283    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:44.406461 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:44.406418    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv" podStartSLOduration=2.406404436 podStartE2EDuration="2.406404436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:56:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:56:44.404646422 +0000 UTC m=+2220.728866944" watchObservedRunningTime="2026-09-09 12:56:44.406404436 +0000 UTC m=+2220.730624967"
Sep 09 12:56:44.589560 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:44.589532    2579 scope.go:117] "RemoveContainer" containerID="04503d7587b0945a202db16446899a04e5623183d50db1f56432a9923f7d67ac"
Sep 09 12:56:44.597093 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:44.597076    2579 scope.go:117] "RemoveContainer" containerID="03bf9237747212cac36932332f8b4e3fc9212acff55371c1c4e6bdff7961a5c0"
Sep 09 12:56:45.304549 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:45.304519    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9vslz_0e58e54d-f883-4fbb-8ddf-b8f65dc39874/dns/0.log"
Sep 09 12:56:45.325714 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:45.325684    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9vslz_0e58e54d-f883-4fbb-8ddf-b8f65dc39874/kube-rbac-proxy/0.log"
Sep 09 12:56:45.476287 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:45.476253    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mnqhx_d47343aa-225c-4858-810e-414e24704ba5/dns-node-resolver/0.log"
Sep 09 12:56:46.086124 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:46.086080    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qlcww_f945ef8e-784f-4df0-83c3-b1597dbe942d/node-ca/0.log"
Sep 09 12:56:47.138570 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:47.138534    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799czcjgf_8da005ca-91cf-4a00-a21b-7c1291a79f29/istio-proxy/0.log"
Sep 09 12:56:47.553575 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:47.553526    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-n9cd4_3f56633f-41e1-4539-af32-8462cf7576de/discovery/0.log"
Sep 09 12:56:47.687157 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:47.687118    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2zjb7_5c881f1f-f3ec-4752-834e-2a694ab52ce2/payload-pre-processing/0.log"
Sep 09 12:56:48.636123 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:48.636095    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j46h4_949d2d6e-2a7a-4d2b-9d52-9303f3540d23/serve-healthcheck-canary/0.log"
Sep 09 12:56:49.173379 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:49.173347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftk7f_d4f4314f-3c74-415b-8223-05252a5b04db/kube-rbac-proxy/0.log"
Sep 09 12:56:49.194342 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:49.194314    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftk7f_d4f4314f-3c74-415b-8223-05252a5b04db/exporter/0.log"
Sep 09 12:56:49.219196 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:49.219167    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftk7f_d4f4314f-3c74-415b-8223-05252a5b04db/extractor/0.log"
Sep 09 12:56:50.404682 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:50.404619    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-prpjx/perf-node-gather-daemonset-pjbmv"
Sep 09 12:56:51.801593 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:51.801519    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kxrbl_38b74994-df95-4436-8813-1beba187cf21/server/0.log"
Sep 09 12:56:51.941005 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:51.940962    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j25dc_e33f9672-6676-4ee2-a0c8-7d0185baeeb2/manager/0.log"
Sep 09 12:56:52.025367 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:52.025334    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5d49f968-vmmnf_e589d913-a2d5-42cb-beef-deb4194efcdc/manager/0.log"
Sep 09 12:56:52.048977 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:52.048953    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c5d49f968-vmmnf_e589d913-a2d5-42cb-beef-deb4194efcdc/copy-manifests/0.log"
Sep 09 12:56:53.224958 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:53.224863    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799czcjgf_8da005ca-91cf-4a00-a21b-7c1291a79f29/istio-proxy/0.log"
Sep 09 12:56:53.590978 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:53.590901    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5465bf697c-g486n_7a3e452e-2d73-44de-8a71-0f706c9ad678/manager/0.log"
Sep 09 12:56:53.621401 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:53.621356    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-fxvzd_8c915815-3a94-4eb3-8446-72a02f8d6015/openshift-lws-operator/0.log"
Sep 09 12:56:53.638235 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:53.638203    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-n9cd4_3f56633f-41e1-4539-af32-8462cf7576de/discovery/0.log"
Sep 09 12:56:53.733581 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:53.733547    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2zjb7_5c881f1f-f3ec-4752-834e-2a694ab52ce2/payload-pre-processing/0.log"
Sep 09 12:56:55.239301 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:55.239257    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799czcjgf_8da005ca-91cf-4a00-a21b-7c1291a79f29/istio-proxy/0.log"
Sep 09 12:56:55.658087 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:55.658006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-n9cd4_3f56633f-41e1-4539-af32-8462cf7576de/discovery/0.log"
Sep 09 12:56:55.768047 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:55.768015    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-2zjb7_5c881f1f-f3ec-4752-834e-2a694ab52ce2/payload-pre-processing/0.log"
Sep 09 12:56:58.644333 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:58.644302    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-q2rvz_4603b301-6bd9-4618-83c9-af2d40a95383/migrator/0.log"
Sep 09 12:56:58.664545 ip-10-0-140-93 kubenswrapper[2579]: I0909 12:56:58.664516    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-q2rvz_4603b301-6bd9-4618-83c9-af2d40a95383/graceful-termination/0.log"