Sep 09 20:46:38.790253 ip-10-0-129-13 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 20:46:38.790267 ip-10-0-129-13 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 20:46:38.790275 ip-10-0-129-13 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 20:46:38.790509 ip-10-0-129-13 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 20:46:48.965505 ip-10-0-129-13 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 20:46:48.965521 ip-10-0-129-13 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 89ae441fa3af441280bdbab6b7c958e0 --
Sep 09 20:48:25.921402 ip-10-0-129-13 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 20:48:26.346706 ip-10-0-129-13 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:48:26.346706 ip-10-0-129-13 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 20:48:26.346706 ip-10-0-129-13 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:48:26.346706 ip-10-0-129-13 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 20:48:26.346706 ip-10-0-129-13 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 20:48:26.347785 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.347696    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 20:48:26.352834 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352814    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:48:26.352834 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352833    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:48:26.352834 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352836    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352840    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352843    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352845    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352848    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352851    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352853    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352857    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352860    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352862    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352865    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352883    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352893    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352896    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352899    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352902    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352905    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352907    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352910    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352913    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:48:26.352928 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352915    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352918    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352922    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352924    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352927    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352930    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352934    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352938    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352941    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352944    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352946    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352949    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352951    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352954    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352956    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352959    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352961    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352963    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352966    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:48:26.353412 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352968    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352971    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352973    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352976    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352978    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352981    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352983    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352986    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352988    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352991    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352993    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.352998    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353002    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353005    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353007    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353010    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353013    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353015    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353018    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353021    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:48:26.354061 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353023    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353025    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353028    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353030    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353032    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353035    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353037    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353039    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353042    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353044    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353046    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353050    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353053    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353056    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353059    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353061    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353064    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353067    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353070    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:48:26.354844 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353073    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353089    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353092    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353095    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353098    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353101    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353104    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353107    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353109    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353113    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353115    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353118    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353120    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353123    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353125    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353128    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353130    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353133    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.353135    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355025    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:48:26.355380 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355176    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355183    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355186    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355189    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355193    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355196    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355199    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355202    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355204    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355207    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355210    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355213    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355215    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355218    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355220    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355223    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355225    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355228    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355231    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355234    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:48:26.355864 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355236    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355239    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355242    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355244    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355247    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355249    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355252    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355254    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355256    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355259    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355266    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355269    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355272    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355275    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355277    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355282    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355286    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355290    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355293    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355295    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:48:26.356391 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355298    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355301    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355306    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355311    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355314    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355317    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355320    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355323    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355325    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355329    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355332    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355335    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355339    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355342    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355344    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355347    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355349    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355352    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355355    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:48:26.356895 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355358    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355360    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355363    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355366    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355368    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355371    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355374    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355377    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355379    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355382    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355385    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355387    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355390    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355393    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355398    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355401    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355406    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355409    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355412    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:48:26.357381 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355414    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355417    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355419    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355422    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355425    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355428    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355430    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355433    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355435    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355438    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355440    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355443    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355445    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355447    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355450    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355452    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355455    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355457    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355460    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355463    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:48:26.357843 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355885    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355891    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355895    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355899    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355902    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355905    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355909    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355911    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355914    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355917    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355920    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355922    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355925    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355927    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355930    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355933    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355936    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355938    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355941    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:48:26.358357 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355943    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355946    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355948    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355951    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355953    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355956    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355958    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355961    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355963    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355965    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355968    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355971    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355973    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355976    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355979    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355981    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355984    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355986    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355989    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355994    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:48:26.358824 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.355997    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356000    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356003    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356006    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356009    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356011    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356014    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356016    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356019    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356022    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356025    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356027    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356030    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356032    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356035    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356037    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356039    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356042    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356044    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356047    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:48:26.359317 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356049    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356052    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356054    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356057    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356060    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356063    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356065    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356068    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356071    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356091    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356096    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356099    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356101    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356104    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356107    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356110    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356113    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356115    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356117    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:48:26.359804 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356120    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356122    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356126    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356129    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356134    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356137    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356139    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356142    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356144    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356147    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356149    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356151    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356154    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356157    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356159    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356162    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356165    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356167    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356170    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:48:26.360300 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356172    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356174    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356244    2571 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356286    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356293    2571 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356298    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356305    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356308    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356313    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356317    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356320    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356323    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356327    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356330    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356333    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356336    2571 flags.go:64] FLAG: --cgroup-root=""
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356339    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356342    2571 flags.go:64] FLAG: --client-ca-file=""
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356346    2571 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356349    2571 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356354    2571 flags.go:64] FLAG: --cluster-domain=""
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356357    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356360    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356363    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 20:48:26.360859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356366    2571 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356370    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356373    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356376    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356380    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356383    2571 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356386    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356389    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356392    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356396    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356400    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356403    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356406    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356409    2571 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356412    2571 flags.go:64] FLAG: --enable-server="true"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356415    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356420    2571 flags.go:64] FLAG: --event-burst="100"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356423    2571 flags.go:64] FLAG: --event-qps="50"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356425    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356428    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356431    2571 flags.go:64] FLAG: --eviction-hard=""
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356435    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356438    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356441    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356444    2571 flags.go:64] FLAG: --eviction-soft=""
Sep 09 20:48:26.361448 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356446    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356449    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356453    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356456    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356459    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356462    2571 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356465    2571 flags.go:64] FLAG: --feature-gates=""
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356468    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356472    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356475    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356478    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356481    2571 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356484    2571 flags.go:64] FLAG: --help="false"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356487    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356490    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356493    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356496    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356499    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356502    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356505    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356508    2571 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356510    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356513    2571 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 20:48:26.362037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356517    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356520    2571 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356522    2571 flags.go:64] FLAG: --kube-reserved=""
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356525    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356528    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356531    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356534    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356536    2571 flags.go:64] FLAG: --lock-file=""
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356539    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356542    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356545    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356550    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356553    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356566    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356570    2571 flags.go:64] FLAG: --logging-format="text"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356572    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356576    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356579    2571 flags.go:64] FLAG: --manifest-url=""
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356582    2571 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356587    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356590    2571 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356594    2571 flags.go:64] FLAG: --max-pods="110"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356597    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356600    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356603    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 20:48:26.362639 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356606    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356609    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356612    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356614    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356622    2571 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356625    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356628    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356631    2571 flags.go:64] FLAG: --pod-cidr=""
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356634    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356640    2571 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356643    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356645    2571 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356648    2571 flags.go:64] FLAG: --port="10250"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356651    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356654    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0134cff6e5d176730"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356657    2571 flags.go:64] FLAG: --qos-reserved=""
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356660    2571 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356663    2571 flags.go:64] FLAG: --register-node="true"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356666    2571 flags.go:64] FLAG: --register-with-taints=""
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356673    2571 flags.go:64] FLAG: --registry-burst="10"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356680    2571 flags.go:64] FLAG: --registry-qps="5"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356683    2571 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356687    2571 flags.go:64] FLAG: --reserved-memory=""
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356690    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356693    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 20:48:26.363269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356698    2571 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356701    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356703    2571 flags.go:64] FLAG: --runonce="false"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356706    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356709    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356712    2571 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356715    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356718    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356721    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356724    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356726    2571 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356729    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356732    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356735    2571 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356737    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356740    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356743    2571 flags.go:64] FLAG: --system-cgroups=""
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356746    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356751    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356754    2571 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356756    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356761    2571 flags.go:64] FLAG: --tls-min-version=""
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356764    2571 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356766    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356769    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 20:48:26.363911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356772    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356776    2571 flags.go:64] FLAG: --v="2"
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356780    2571 flags.go:64] FLAG: --version="false"
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356785    2571 flags.go:64] FLAG: --vmodule=""
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356789    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.356792    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356900    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356905    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356909    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356911    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356914    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356917    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356919    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356922    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356924    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356927    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356930    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356932    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356935    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356937    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356940    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:48:26.364534 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356943    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356947    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356950    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356954    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356956    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356959    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356962    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356964    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356966    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356969    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356971    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356974    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356979    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356981    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356984    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356987    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356989    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356991    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356995    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:48:26.365055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.356998    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357000    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357002    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357005    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357009    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357012    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357015    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357018    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357020    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357023    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357025    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357028    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357030    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357033    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357035    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357038    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357040    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357043    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357045    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:48:26.365624 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357049    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357051    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357054    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357056    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357058    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357061    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357065    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357068    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357070    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357073    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357095    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357099    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357102    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357105    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357108    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357110    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357113    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357115    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357118    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357120    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:48:26.366111 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357122    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357125    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357128    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357130    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357132    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357135    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357138    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357140    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357143    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357145    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357148    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357150    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357153    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357155    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357158    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357160    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357163    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357165    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357169    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357172    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:48:26.366609 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357174    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357177    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357179    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357182    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357186    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.357188    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.357830    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.365845    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.365862    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.365883    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.365888    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365947    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365951    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365955    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:48:26.367153 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365958    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365961    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365964    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365967    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365969    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365973    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365975    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365978    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365981    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365983    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365988    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365992    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365995    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.365998    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366001    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366004    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366007    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366010    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366012    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:48:26.367509 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366015    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366018    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366021    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366023    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366026    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366028    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366031    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366039    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366042    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366044    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366047    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366050    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366052    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366055    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366058    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366061    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366064    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366068    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366071    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:48:26.367966 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366091    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366095    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366098    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366100    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366104    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366109    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366112    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366115    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366118    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366121    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366123    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366126    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366128    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366131    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366133    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366136    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366138    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366141    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366144    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366146    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:48:26.368513 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366149    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366153    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366156    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366158    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366161    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366163    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366166    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366168    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366171    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366173    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366176    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366178    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366180    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366183    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366185    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366188    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366190    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366193    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366195    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366198    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:48:26.369016 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366200    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366203    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366205    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366208    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366210    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366213    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366215    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366218    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366220    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366222    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366225    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366227    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366231    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366233    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366237    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366240    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366243    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:48:26.369543 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366245    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.366250    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366369    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366374    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366377    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366380    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366383    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366386    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366388    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366391    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366393    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366395    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366398    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366400    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 20:48:26.369962 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366403    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366405    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366408    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366410    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366412    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366415    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366417    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366419    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366423    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366427    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366430    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366433    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366435    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366438    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366441    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366444    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366447    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366449    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366452    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366455    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 20:48:26.370324 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366457    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366460    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366462    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366464    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366467    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366470    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366473    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366476    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366479    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366482    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366485    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366487    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366490    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366493    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366495    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366498    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366500    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366504    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366507    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 20:48:26.370808 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366510    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366513    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366516    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366519    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366521    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366524    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366527    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366529    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366532    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366534    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366537    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366540    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366542    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366545    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366547    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366549    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366552    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366554    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366557    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366559    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 20:48:26.371286 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366561    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366564    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366566    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366569    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366571    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366573    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366576    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366578    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366581    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366583    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366586    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366589    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366591    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366593    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366596    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366599    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366601    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366603    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366606    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366609    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 20:48:26.371764 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366612    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366614    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366617    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366620    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366623    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366625    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366627    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:26.366630    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.366635    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.366768    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.369237    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.370352    2571 server.go:1019] "Starting client certificate rotation"
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.370450    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:48:26.372335 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.371429    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 20:48:26.373359 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.373344    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:48:26.374511 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.374493    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 20:48:26.390110 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.390060    2571 log.go:25] "Validated CRI v1 runtime API"
Sep 09 20:48:26.395854 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.395835    2571 log.go:25] "Validated CRI v1 image API"
Sep 09 20:48:26.396755 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.396736    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 20:48:26.402328 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.402285    2571 fs.go:135] Filesystem UUIDs: map[3bb5e722-6616-4085-821f-37bd1088ea54:/dev/nvme0n1p3 3c74e28a-0027-4411-a5eb-9b4c3a838c80:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 09 20:48:26.402415 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.402322    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 20:48:26.402918 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.402895    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:48:26.407808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.407683    2571 manager.go:216] Machine: {Timestamp:2026-09-09 20:48:26.406231752 +0000 UTC m=+0.376798461 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097145 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b281e9c5bdc79aba16e58d53d34af SystemUUID:ec2b281e-9c5b-dc79-aba1-6e58d53d34af BootID:89ae441f-a3af-4412-80bd-bab6b7c958e0 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a4:08:d4:47:e5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a4:08:d4:47:e5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:6c:0f:f3:85:a8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 20:48:26.407808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.407800    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 20:48:26.407964 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.407891    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 20:48:26.411523 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.411492    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 20:48:26.411705 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.411527    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-13.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 20:48:26.413162 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.413148    2571 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 20:48:26.413227 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.413169    2571 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 20:48:26.413227 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.413180    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:48:26.414054 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.414041    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 20:48:26.416029 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.416016    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 20:48:26.420658 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.420642    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:48:26.420804 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.420793    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 20:48:26.420858 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.420836    2571 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 20:48:26.420858 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.420852    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 20:48:26.420937 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.420873    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 20:48:26.420937 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.420887    2571 kubelet.go:398] "Adding apiserver pod source"
Sep 09 20:48:26.420937 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.420901    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 20:48:26.421321 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.421308    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:48:26.421401 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.421332    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 20:48:26.423782 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.423763    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 20:48:26.426773 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.426758    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 20:48:26.426833 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.426793    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 20:48:26.426985 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.426975    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 20:48:26.427018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.426991    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 20:48:26.427018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.426997    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 20:48:26.427018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427002    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 20:48:26.427018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427010    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 20:48:26.427018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427015    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 20:48:26.427642 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427632    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 20:48:26.427678 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427643    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 20:48:26.427678 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427651    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 20:48:26.427678 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427656    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 20:48:26.427678 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427665    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 20:48:26.427678 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.427673    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 20:48:26.428610 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.428599    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 20:48:26.428610 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.428609    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 20:48:26.431149 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.431128    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-13.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 20:48:26.431889 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.431866    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 20:48:26.431953 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.431867    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 20:48:26.432590 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.432577    2571 server.go:1268] "Started kubelet"
Sep 09 20:48:26.432717 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.432691    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 20:48:26.432768 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.432689    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 20:48:26.432768 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.432763    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 20:48:26.433616 ip-10-0-129-13 systemd[1]: Started Kubernetes Kubelet.
Sep 09 20:48:26.434062 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.433796    2571 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 20:48:26.438830 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.438807    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 20:48:26.441130 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.441069    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 20:48:26.443343 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.442018    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-13.ec2.internal.18d3c21ce9bc98cf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-13.ec2.internal,UID:ip-10-0-129-13.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-13.ec2.internal,},FirstTimestamp:2026-09-09 20:48:26.432542927 +0000 UTC m=+0.403109634,LastTimestamp:2026-09-09 20:48:26.432542927 +0000 UTC m=+0.403109634,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-13.ec2.internal,}"
Sep 09 20:48:26.445806 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.445789    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 20:48:26.446546 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.446517    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 20:48:26.446876 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.446617    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:26.447343 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447322    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 20:48:26.447343 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447325    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 20:48:26.447506 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447356    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 20:48:26.447506 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447366    2571 factory.go:55] Registering systemd factory
Sep 09 20:48:26.447506 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447402    2571 factory.go:223] Registration of the systemd container factory successfully
Sep 09 20:48:26.447506 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447457    2571 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 20:48:26.447506 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447468    2571 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 20:48:26.447741 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447717    2571 factory.go:153] Registering CRI-O factory
Sep 09 20:48:26.447809 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447743    2571 factory.go:223] Registration of the crio container factory successfully
Sep 09 20:48:26.447809 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447791    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 20:48:26.447809 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447809    2571 factory.go:103] Registering Raw factory
Sep 09 20:48:26.447955 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.447824    2571 manager.go:1196] Started watching for new ooms in manager
Sep 09 20:48:26.448204 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.448187    2571 manager.go:349] Starting recovery of all containers
Sep 09 20:48:26.453681 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.453546    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f9hsv"
Sep 09 20:48:26.457547 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.457524    2571 manager.go:354] Recovery completed
Sep 09 20:48:26.458349 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.458320    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 20:48:26.458349 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.458321    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-13.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 20:48:26.459676 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.459643    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-f9hsv"
Sep 09 20:48:26.461896 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.461884    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:48:26.464317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.464298    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:48:26.464392 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.464329    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:48:26.464392 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.464340    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:48:26.464944 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.464927    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 20:48:26.464944 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.464943    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 20:48:26.465071 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.464957    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 20:48:26.465071 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.464928    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:48:26.466349 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.466334    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:48:26.466419 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.466364    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:48:26.466419 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.466377    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:48:26.466963 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.466952    2571 policy_none.go:49] "None policy: Start"
Sep 09 20:48:26.467006 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.466968    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 20:48:26.467006 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.466978    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 20:48:26.469040 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.469028    2571 policy_none.go:47] "Start"
Sep 09 20:48:26.502659 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.502642    2571 manager.go:337] "Starting Device Plugin manager"
Sep 09 20:48:26.502755 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.502685    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 20:48:26.502755 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.502695    2571 server.go:87] "Starting device plugin registration server"
Sep 09 20:48:26.503045 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.503033    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 20:48:26.503094 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.503048    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 20:48:26.503300 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.503163    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 20:48:26.503300 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.503240    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 20:48:26.503300 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.503250    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 20:48:26.504153 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.504134    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 20:48:26.504277 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.504185    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:26.557759 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.557724    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 20:48:26.560013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.558958    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 20:48:26.560013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.559007    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 20:48:26.560013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.559029    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 20:48:26.560013 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.559069    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 20:48:26.561294 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.561276    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:48:26.604281 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.604179    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:48:26.605121 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.605103    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:48:26.605177 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.605142    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:48:26.605177 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.605157    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:48:26.605236 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.605186    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.613896 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.613876    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.613950 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.613898    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-13.ec2.internal\": node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:26.641702 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.641679    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:26.659156 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.659132    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"]
Sep 09 20:48:26.659219 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.659204    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:48:26.660175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.660160    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:48:26.660222 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.660190    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:48:26.660222 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.660200    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:48:26.662483 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.662471    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:48:26.662597 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.662582    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.662634 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.662612    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:48:26.663202 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.663185    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:48:26.663290 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.663220    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:48:26.663290 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.663235    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:48:26.663290 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.663187    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:48:26.663290 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.663285    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:48:26.663467 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.663296    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:48:26.665591 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.665573    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.665652 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.665609    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 20:48:26.666264 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.666248    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 20:48:26.666337 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.666275    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 20:48:26.666337 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.666290    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeHasSufficientPID"
Sep 09 20:48:26.695701 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.695671    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-13.ec2.internal\" not found" node="ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.699945 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.699927    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-13.ec2.internal\" not found" node="ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.742500 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.742477    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:26.748935 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.748915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bed195b806fdc3af1a773cfd2f90a22f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-13.ec2.internal\" (UID: \"bed195b806fdc3af1a773cfd2f90a22f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.748990 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.748940    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/37da337e78d8a9155f29cc132c49e73a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal\" (UID: \"37da337e78d8a9155f29cc132c49e73a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.748990 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.748962    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37da337e78d8a9155f29cc132c49e73a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal\" (UID: \"37da337e78d8a9155f29cc132c49e73a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.843136 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.843097    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:26.849450 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.849426    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bed195b806fdc3af1a773cfd2f90a22f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-13.ec2.internal\" (UID: \"bed195b806fdc3af1a773cfd2f90a22f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.849502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.849460    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/37da337e78d8a9155f29cc132c49e73a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal\" (UID: \"37da337e78d8a9155f29cc132c49e73a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.849502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.849478    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37da337e78d8a9155f29cc132c49e73a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal\" (UID: \"37da337e78d8a9155f29cc132c49e73a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.849562 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.849536    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37da337e78d8a9155f29cc132c49e73a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal\" (UID: \"37da337e78d8a9155f29cc132c49e73a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.849562 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.849542    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/37da337e78d8a9155f29cc132c49e73a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal\" (UID: \"37da337e78d8a9155f29cc132c49e73a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.849626 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.849536    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bed195b806fdc3af1a773cfd2f90a22f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-13.ec2.internal\" (UID: \"bed195b806fdc3af1a773cfd2f90a22f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:26.943928 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:26.943902    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:26.998485 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:26.998453    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:27.002201 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.002182    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:27.044841 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.044804    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:27.145424 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.145390    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:27.245971 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.245897    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-13.ec2.internal\" not found"
Sep 09 20:48:27.254748 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.254730    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:48:27.346776 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.346736    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:27.354721 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.354694    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 20:48:27.356144 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.356126    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:27.369662 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.369635    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 20:48:27.369793 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.369771    2571 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://ac092e7aac7aa4b64a003d449c137ad2-4d400d18f2e19090.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.129.13:44500->23.20.197.8:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal"
Sep 09 20:48:27.369855 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.369799    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 20:48:27.386846 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.386813    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:48:27.421507 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.421463    2571 apiserver.go:52] "Watching apiserver"
Sep 09 20:48:27.427386 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.427363    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 20:48:27.428153 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.428132    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-jffdc","openshift-dns/node-resolver-nlzcr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal","openshift-multus/multus-additional-cni-plugins-7j67z","openshift-network-diagnostics/network-check-target-n7th9","openshift-network-operator/iptables-alerter-5zn5j","openshift-ovn-kubernetes/ovnkube-node-wsth2","kube-system/global-pull-secret-syncer-6b6sc","kube-system/konnectivity-agent-6lkd2","kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr","openshift-cluster-node-tuning-operator/tuned-bw5dh","openshift-image-registry/node-ca-rp6l9","openshift-multus/multus-kff9p"]
Sep 09 20:48:27.430559 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.430538    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.432608 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.432580    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 20:48:27.432796 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.432777    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 20:48:27.432940 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.432917    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 20:48:27.433406 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.433347    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 20:48:27.433493 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.433355    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 20:48:27.433493 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.433392    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rjpjf\""
Sep 09 20:48:27.433685 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.433502    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 20:48:27.433741 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.433680    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.435738 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.435703    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 20:48:27.435864 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.435839    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 20:48:27.435988 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.435917    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 20:48:27.436426 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.436398    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.436764 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.436580    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zbbcc\""
Sep 09 20:48:27.438861 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.438677    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 20:48:27.438861 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.438863    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 20:48:27.439212 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.439194    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qn5k6\""
Sep 09 20:48:27.439411 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.439388    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 20:48:27.439512 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.439442    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 20:48:27.439576 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.439558    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.442525 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.442506    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fhjrn\""
Sep 09 20:48:27.442623 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.442547    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:27.442623 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.442589    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 20:48:27.442733 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.442635    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:27.442969 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.442950    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 20:48:27.444671 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.444652    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.446354 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.446335    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 20:48:27.446781 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.446757    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:48:27.446781 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.446771    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 20:48:27.446928 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.446788    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xzfr2\""
Sep 09 20:48:27.446928 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.446892    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 20:48:27.447181 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.447158    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:27.448967 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.448945    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 20:48:27.449240 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.449221    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 20:48:27.449372 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.449356    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sl4rt\""
Sep 09 20:48:27.449652 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.449635    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:27.449744 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.449723    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:27.451885 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.451865    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.452856 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.452833    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8fc68ca-d113-4505-9b99-7581588403eb-host-slash\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.452934 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.452862    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-systemd\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.452934 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.452879    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-cni-bin\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.452934 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.452892    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:27.453120 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.452946    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-slash\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.453120 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.452980    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.453120 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453038    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-run-ovn-kubernetes\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.453120 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453063    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ndr2\" (UniqueName: \"kubernetes.io/projected/99e6a99b-0004-4de5-86c5-cd1c5a747f75-kube-api-access-5ndr2\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.453120 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8fc68ca-d113-4505-9b99-7581588403eb-iptables-alerter-script\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453138    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-daemon-config\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453161    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-system-cni-dir\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453184    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-run-netns\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453231    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/124b5fb2-9d37-4818-94f7-07af39967174-konnectivity-ca\") pod \"konnectivity-agent-6lkd2\" (UID: \"124b5fb2-9d37-4818-94f7-07af39967174\") " pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453255    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-cni-multus\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-conf-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453327    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-system-cni-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453374 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453362    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-ovn\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453387    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-netns\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453410    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-hostroot\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453433    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-cnibin\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453456    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453481    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x875c\" (UniqueName: \"kubernetes.io/projected/3f86e245-9338-4c00-ae1a-129411488355-kube-api-access-x875c\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453507    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/237b6000-010e-4262-9ed4-9cfd02d1df7d-host\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453543    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/124b5fb2-9d37-4818-94f7-07af39967174-agent-certs\") pod \"konnectivity-agent-6lkd2\" (UID: \"124b5fb2-9d37-4818-94f7-07af39967174\") " pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453581    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-kubelet\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453598    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-cnibin\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453613    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-socket-dir-parent\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453642    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-multus-certs\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453666    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovn-node-metrics-cert\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-node-log\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453696    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq7tv\" (UniqueName: \"kubernetes.io/projected/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-kube-api-access-sq7tv\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:27.453715 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea05bcec-cefd-42ad-968c-81c631ca98a3-cni-binary-copy\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453736    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-var-lib-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453779    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-cni-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453802    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-cni-bin\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453823    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-etc-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453837    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-cni-netd\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453853    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-k8s-cni-cncf-io\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453906    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ljfd\" (UniqueName: \"kubernetes.io/projected/237b6000-010e-4262-9ed4-9cfd02d1df7d-kube-api-access-4ljfd\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453943    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq5jj\" (UniqueName: \"kubernetes.io/projected/f8fc68ca-d113-4505-9b99-7581588403eb-kube-api-access-wq5jj\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.453979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-cni-binary-copy\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454004    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-systemd-units\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454028    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovnkube-script-lib\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454051    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/237b6000-010e-4262-9ed4-9cfd02d1df7d-serviceca\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454163    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-etc-kubernetes\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454218    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454226    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7bhcb\""
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454226    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.454463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454248    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 20:48:27.455018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454268    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovnkube-config\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.455018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454289    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-env-overrides\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.455018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454350    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:27.455018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454467    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj69p\" (UniqueName: \"kubernetes.io/projected/ea05bcec-cefd-42ad-968c-81c631ca98a3-kube-api-access-qj69p\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.455018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-kubelet\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.455018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-log-socket\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.455018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.454686    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.455018 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.454818    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:27.457152 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.457127    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.458325 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.458305    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 20:48:27.459295 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.459267    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 20:48:27.459388 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.459307    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 20:48:27.459441 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.459412    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6jhxf\""
Sep 09 20:48:27.459496 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.459413    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 20:48:27.459752 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.459735    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.462711 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.462382    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 20:43:27 +0000 UTC" deadline="2028-03-21 11:16:48.461168202 +0000 UTC"
Sep 09 20:48:27.462711 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.462447    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13406h28m20.998725105s"
Sep 09 20:48:27.464765 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.463484    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w2f6z\""
Sep 09 20:48:27.464765 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.463814    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:48:27.464765 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.464041    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 20:48:27.482211 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.482181    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7h7l9"
Sep 09 20:48:27.489991 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.489969    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7h7l9"
Sep 09 20:48:27.547938 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.547913    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 20:48:27.555129 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555099    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76649ca0-30fa-40ea-8854-2ddab01b1692-tmp-dir\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.555129 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-modprobe-d\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.555317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555153    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:27.555317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555188    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj69p\" (UniqueName: \"kubernetes.io/projected/ea05bcec-cefd-42ad-968c-81c631ca98a3-kube-api-access-qj69p\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.555317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555213    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-kubelet\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555229    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-log-socket\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555247    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.555317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.555317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555294    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysctl-d\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.555317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555302    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-log-socket\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555309    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-kubelet\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555334    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8fc68ca-d113-4505-9b99-7581588403eb-host-slash\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555352    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-systemd\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-cni-bin\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555400    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8fc68ca-d113-4505-9b99-7581588403eb-host-slash\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555420    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysconfig\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555442    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-host\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555448    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-systemd\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555452    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-cni-bin\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555470    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555495    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9efdb610-3597-487e-91c0-b0e85263d225-kubelet-config\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.555552    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555554    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-slash\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.555596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555598    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-slash\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.555619    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:48:28.055597604 +0000 UTC m=+2.026164300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555658    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555686    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-sys-fs\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555709    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-run-ovn-kubernetes\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555731    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ndr2\" (UniqueName: \"kubernetes.io/projected/99e6a99b-0004-4de5-86c5-cd1c5a747f75-kube-api-access-5ndr2\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555755    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk4zx\" (UniqueName: \"kubernetes.io/projected/f1716ba5-d90b-4532-ad98-9a0865c4a39c-kube-api-access-xk4zx\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555762    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555787    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76649ca0-30fa-40ea-8854-2ddab01b1692-hosts-file\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555814    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-run-ovn-kubernetes\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555842    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8fc68ca-d113-4505-9b99-7581588403eb-iptables-alerter-script\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555868    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-daemon-config\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555883    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-system-cni-dir\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555897    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-run-netns\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555947    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/124b5fb2-9d37-4818-94f7-07af39967174-konnectivity-ca\") pod \"konnectivity-agent-6lkd2\" (UID: \"124b5fb2-9d37-4818-94f7-07af39967174\") " pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:27.556132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555963    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-system-cni-dir\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.555978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-cni-multus\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556004    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-conf-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556012    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-run-netns\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-system-cni-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556064    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-cni-multus\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-ovn\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556123    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-lib-modules\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556147    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-netns\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556148    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556171    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-hostroot\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556203    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-system-cni-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556222    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-run-ovn\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556226    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-cnibin\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556250    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-netns\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556265    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3f86e245-9338-4c00-ae1a-129411488355-cnibin\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556272    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556297    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-hostroot\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.556957 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x875c\" (UniqueName: \"kubernetes.io/projected/3f86e245-9338-4c00-ae1a-129411488355-kube-api-access-x875c\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556326    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/237b6000-010e-4262-9ed4-9cfd02d1df7d-host\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556351    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-device-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556378    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc6g7\" (UniqueName: \"kubernetes.io/projected/84d06660-1020-464f-93f3-084afba5ab9d-kube-api-access-zc6g7\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556402    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/124b5fb2-9d37-4818-94f7-07af39967174-agent-certs\") pod \"konnectivity-agent-6lkd2\" (UID: \"124b5fb2-9d37-4818-94f7-07af39967174\") " pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556430    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-kubelet\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556455    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98h8x\" (UniqueName: \"kubernetes.io/projected/76649ca0-30fa-40ea-8854-2ddab01b1692-kube-api-access-98h8x\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556478    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysctl-conf\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556306    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-conf-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556503    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/84d06660-1020-464f-93f3-084afba5ab9d-etc-tuned\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556572    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/237b6000-010e-4262-9ed4-9cfd02d1df7d-host\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556579    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-daemon-config\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556576    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-kubelet\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556610    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/124b5fb2-9d37-4818-94f7-07af39967174-konnectivity-ca\") pod \"konnectivity-agent-6lkd2\" (UID: \"124b5fb2-9d37-4818-94f7-07af39967174\") " pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556676    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8fc68ca-d113-4505-9b99-7581588403eb-iptables-alerter-script\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556700    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-cnibin\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-socket-dir-parent\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556750    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-cnibin\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.557808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556764    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-multus-certs\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovn-node-metrics-cert\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556803    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-socket-dir-parent\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556815    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-node-log\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556839    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556841    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq7tv\" (UniqueName: \"kubernetes.io/projected/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-kube-api-access-sq7tv\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556878    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-node-log\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea05bcec-cefd-42ad-968c-81c631ca98a3-cni-binary-copy\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556842    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-multus-certs\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556910    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556930    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-var-lib-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556947    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-registration-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556967    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9efdb610-3597-487e-91c0-b0e85263d225-dbus\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.556992    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-cni-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557015    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-cni-bin\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557037    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-etc-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557073    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-cni-netd\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.558563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557101    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-var-lib-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557127    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-multus-cni-dir\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557118    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-socket-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557163    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-etc-openvswitch\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557169    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-systemd\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557184    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-var-lib-cni-bin\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557210    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-cni-netd\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557197    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-sys\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557243    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-var-lib-kubelet\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557270    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-k8s-cni-cncf-io\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557296    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ljfd\" (UniqueName: \"kubernetes.io/projected/237b6000-010e-4262-9ed4-9cfd02d1df7d-kube-api-access-4ljfd\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557331    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq5jj\" (UniqueName: \"kubernetes.io/projected/f8fc68ca-d113-4505-9b99-7581588403eb-kube-api-access-wq5jj\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557380    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-host-run-k8s-cni-cncf-io\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557387    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557432    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-cni-binary-copy\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557469    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-systemd-units\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557494    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovnkube-script-lib\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557517    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/237b6000-010e-4262-9ed4-9cfd02d1df7d-serviceca\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557544    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-kubernetes\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557493    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea05bcec-cefd-42ad-968c-81c631ca98a3-cni-binary-copy\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557569    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-run\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557592    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84d06660-1020-464f-93f3-084afba5ab9d-tmp\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557616    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-etc-kubernetes\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557554    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-systemd-units\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557641    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovnkube-config\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557665    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557690    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-env-overrides\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557715    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557690    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea05bcec-cefd-42ad-968c-81c631ca98a3-etc-kubernetes\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.557975    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/237b6000-010e-4262-9ed4-9cfd02d1df7d-serviceca\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.558009    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99e6a99b-0004-4de5-86c5-cd1c5a747f75-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.558154    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-env-overrides\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.558175    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovnkube-script-lib\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.558207    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovnkube-config\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.559687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.558655    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3f86e245-9338-4c00-ae1a-129411488355-cni-binary-copy\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.561310 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.561289    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/99e6a99b-0004-4de5-86c5-cd1c5a747f75-ovn-node-metrics-cert\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.561400 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.561382    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/124b5fb2-9d37-4818-94f7-07af39967174-agent-certs\") pod \"konnectivity-agent-6lkd2\" (UID: \"124b5fb2-9d37-4818-94f7-07af39967174\") " pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:27.564823 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.564795    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:48:27.564823 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.564821    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:48:27.564968 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.564835    2571 projected.go:196] Error preparing data for projected volume kube-api-access-2r5pw for pod openshift-network-diagnostics/network-check-target-n7th9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:27.564968 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.564911    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw podName:4b0a6d42-8655-44e9-b30b-301aca1d6115 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:28.064893011 +0000 UTC m=+2.035459723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2r5pw" (UniqueName: "kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw") pod "network-check-target-n7th9" (UID: "4b0a6d42-8655-44e9-b30b-301aca1d6115") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:27.565261 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.565238    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj69p\" (UniqueName: \"kubernetes.io/projected/ea05bcec-cefd-42ad-968c-81c631ca98a3-kube-api-access-qj69p\") pod \"multus-kff9p\" (UID: \"ea05bcec-cefd-42ad-968c-81c631ca98a3\") " pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.566879 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.566854    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x875c\" (UniqueName: \"kubernetes.io/projected/3f86e245-9338-4c00-ae1a-129411488355-kube-api-access-x875c\") pod \"multus-additional-cni-plugins-7j67z\" (UID: \"3f86e245-9338-4c00-ae1a-129411488355\") " pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.567022 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.566992    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ndr2\" (UniqueName: \"kubernetes.io/projected/99e6a99b-0004-4de5-86c5-cd1c5a747f75-kube-api-access-5ndr2\") pod \"ovnkube-node-wsth2\" (UID: \"99e6a99b-0004-4de5-86c5-cd1c5a747f75\") " pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.567385 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.567365    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq7tv\" (UniqueName: \"kubernetes.io/projected/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-kube-api-access-sq7tv\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:27.568070 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.568053    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq5jj\" (UniqueName: \"kubernetes.io/projected/f8fc68ca-d113-4505-9b99-7581588403eb-kube-api-access-wq5jj\") pod \"iptables-alerter-5zn5j\" (UID: \"f8fc68ca-d113-4505-9b99-7581588403eb\") " pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.568338 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.568317    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ljfd\" (UniqueName: \"kubernetes.io/projected/237b6000-010e-4262-9ed4-9cfd02d1df7d-kube-api-access-4ljfd\") pod \"node-ca-rp6l9\" (UID: \"237b6000-010e-4262-9ed4-9cfd02d1df7d\") " pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.587474 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.587435    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37da337e78d8a9155f29cc132c49e73a.slice/crio-920dd7f005204d90f30c4fbe657504693e147b338dc19381948eab6b168cb7b7 WatchSource:0}: Error finding container 920dd7f005204d90f30c4fbe657504693e147b338dc19381948eab6b168cb7b7: Status 404 returned error can't find the container with id 920dd7f005204d90f30c4fbe657504693e147b338dc19381948eab6b168cb7b7
Sep 09 20:48:27.587745 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.587725    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed195b806fdc3af1a773cfd2f90a22f.slice/crio-70271572febf830cc5ef30b06036c7c520ab9aaa660b3b509aabb522ec60a36f WatchSource:0}: Error finding container 70271572febf830cc5ef30b06036c7c520ab9aaa660b3b509aabb522ec60a36f: Status 404 returned error can't find the container with id 70271572febf830cc5ef30b06036c7c520ab9aaa660b3b509aabb522ec60a36f
Sep 09 20:48:27.595365 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.595340    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:48:27.658185 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658144    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-device-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.658185 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc6g7\" (UniqueName: \"kubernetes.io/projected/84d06660-1020-464f-93f3-084afba5ab9d-kube-api-access-zc6g7\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658207    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98h8x\" (UniqueName: \"kubernetes.io/projected/76649ca0-30fa-40ea-8854-2ddab01b1692-kube-api-access-98h8x\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.658423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658228    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysctl-conf\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658252    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/84d06660-1020-464f-93f3-084afba5ab9d-etc-tuned\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658272    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-device-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.658423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658281    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-registration-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.658423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9efdb610-3597-487e-91c0-b0e85263d225-dbus\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.658423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658396    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-registration-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.658423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658412    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-socket-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658442    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-systemd\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658465    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-sys\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658441    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9efdb610-3597-487e-91c0-b0e85263d225-dbus\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658488    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-var-lib-kubelet\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658418    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysctl-conf\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658503    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-systemd\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658526    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-sys\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658527    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-socket-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658556    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-var-lib-kubelet\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658555    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-kubernetes\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658580    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-run\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658644    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-kubernetes\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658654    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84d06660-1020-464f-93f3-084afba5ab9d-tmp\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658695    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658700    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-run\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658720    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76649ca0-30fa-40ea-8854-2ddab01b1692-tmp-dir\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658744    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-modprobe-d\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.658810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658754    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658803    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658829    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658848    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-modprobe-d\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658853    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysctl-d\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658886    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysconfig\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658910    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-host\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658928    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.658936    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658946    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9efdb610-3597-487e-91c0-b0e85263d225-kubelet-config\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysctl-d\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658972    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-sys-fs\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.658982    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-etc-sysconfig\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:27.659003    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret podName:9efdb610-3597-487e-91c0-b0e85263d225 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:28.158983823 +0000 UTC m=+2.129550517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret") pod "global-pull-secret-syncer-6b6sc" (UID: "9efdb610-3597-487e-91c0-b0e85263d225") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659014    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f1716ba5-d90b-4532-ad98-9a0865c4a39c-sys-fs\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659015    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-host\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659037    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9efdb610-3597-487e-91c0-b0e85263d225-kubelet-config\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:27.659537 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659045    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk4zx\" (UniqueName: \"kubernetes.io/projected/f1716ba5-d90b-4532-ad98-9a0865c4a39c-kube-api-access-xk4zx\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.660069 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659071    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76649ca0-30fa-40ea-8854-2ddab01b1692-hosts-file\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.660069 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659035    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76649ca0-30fa-40ea-8854-2ddab01b1692-tmp-dir\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.660069 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659134    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-lib-modules\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.660069 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659147    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/76649ca0-30fa-40ea-8854-2ddab01b1692-hosts-file\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.660069 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.659230    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/84d06660-1020-464f-93f3-084afba5ab9d-lib-modules\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.661259 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.661230    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/84d06660-1020-464f-93f3-084afba5ab9d-etc-tuned\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.661259 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.661256    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84d06660-1020-464f-93f3-084afba5ab9d-tmp\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.667562 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.667542    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98h8x\" (UniqueName: \"kubernetes.io/projected/76649ca0-30fa-40ea-8854-2ddab01b1692-kube-api-access-98h8x\") pod \"node-resolver-nlzcr\" (UID: \"76649ca0-30fa-40ea-8854-2ddab01b1692\") " pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.670316 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.670293    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc6g7\" (UniqueName: \"kubernetes.io/projected/84d06660-1020-464f-93f3-084afba5ab9d-kube-api-access-zc6g7\") pod \"tuned-bw5dh\" (UID: \"84d06660-1020-464f-93f3-084afba5ab9d\") " pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.670417 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.670299    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk4zx\" (UniqueName: \"kubernetes.io/projected/f1716ba5-d90b-4532-ad98-9a0865c4a39c-kube-api-access-xk4zx\") pod \"aws-ebs-csi-driver-node-hdgrr\" (UID: \"f1716ba5-d90b-4532-ad98-9a0865c4a39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.752457 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.752422    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:27.762681 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.762658    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e6a99b_0004_4de5_86c5_cd1c5a747f75.slice/crio-4dda968ee73fb452037a1383b97d291f85a71d9262a155c600501fc6936febd9 WatchSource:0}: Error finding container 4dda968ee73fb452037a1383b97d291f85a71d9262a155c600501fc6936febd9: Status 404 returned error can't find the container with id 4dda968ee73fb452037a1383b97d291f85a71d9262a155c600501fc6936febd9
Sep 09 20:48:27.771903 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.771882    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rp6l9"
Sep 09 20:48:27.782369 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.782339    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kff9p"
Sep 09 20:48:27.790065 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.790045    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea05bcec_cefd_42ad_968c_81c631ca98a3.slice/crio-fcebc201753b9d2d37cdc3fbc8ed11735ce66368a76c13f4baaf4f41092b5c71 WatchSource:0}: Error finding container fcebc201753b9d2d37cdc3fbc8ed11735ce66368a76c13f4baaf4f41092b5c71: Status 404 returned error can't find the container with id fcebc201753b9d2d37cdc3fbc8ed11735ce66368a76c13f4baaf4f41092b5c71
Sep 09 20:48:27.803132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.803112    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7j67z"
Sep 09 20:48:27.812208 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.811964    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f86e245_9338_4c00_ae1a_129411488355.slice/crio-16fda0f4868105f9ff2c0c0cca6693890e664d6e92d648a4c254bbde809123b3 WatchSource:0}: Error finding container 16fda0f4868105f9ff2c0c0cca6693890e664d6e92d648a4c254bbde809123b3: Status 404 returned error can't find the container with id 16fda0f4868105f9ff2c0c0cca6693890e664d6e92d648a4c254bbde809123b3
Sep 09 20:48:27.815989 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.815973    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:48:27.823547 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.823523    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5zn5j"
Sep 09 20:48:27.828697 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.828677    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:27.832252 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.832227    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8fc68ca_d113_4505_9b99_7581588403eb.slice/crio-3d7252f76f828253d4d9fd28f5442e9adf3802cad3fb0eafcf303eefbd18408c WatchSource:0}: Error finding container 3d7252f76f828253d4d9fd28f5442e9adf3802cad3fb0eafcf303eefbd18408c: Status 404 returned error can't find the container with id 3d7252f76f828253d4d9fd28f5442e9adf3802cad3fb0eafcf303eefbd18408c
Sep 09 20:48:27.837572 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.837550    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124b5fb2_9d37_4818_94f7_07af39967174.slice/crio-01a7f10ad149eba16ddb7055f27303310126fdf470ab1d783ecc077f1df8a33a WatchSource:0}: Error finding container 01a7f10ad149eba16ddb7055f27303310126fdf470ab1d783ecc077f1df8a33a: Status 404 returned error can't find the container with id 01a7f10ad149eba16ddb7055f27303310126fdf470ab1d783ecc077f1df8a33a
Sep 09 20:48:27.857713 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.857691    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nlzcr"
Sep 09 20:48:27.869378 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.869348    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76649ca0_30fa_40ea_8854_2ddab01b1692.slice/crio-7fec4053e110ba0703a83e544d24c61654f66e391cab38761a02e84499048b2a WatchSource:0}: Error finding container 7fec4053e110ba0703a83e544d24c61654f66e391cab38761a02e84499048b2a: Status 404 returned error can't find the container with id 7fec4053e110ba0703a83e544d24c61654f66e391cab38761a02e84499048b2a
Sep 09 20:48:27.889896 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.889863    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bw5dh"
Sep 09 20:48:27.891034 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:27.891014    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr"
Sep 09 20:48:27.899803 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:48:27.899777    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d06660_1020_464f_93f3_084afba5ab9d.slice/crio-f47ca923ac6943d33617e7745cb24a06ffdaa2ac6631b861c7ca005893a2d2dd WatchSource:0}: Error finding container f47ca923ac6943d33617e7745cb24a06ffdaa2ac6631b861c7ca005893a2d2dd: Status 404 returned error can't find the container with id f47ca923ac6943d33617e7745cb24a06ffdaa2ac6631b861c7ca005893a2d2dd
Sep 09 20:48:28.063151 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.063040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:28.063303 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.063196    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:28.063303 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.063258    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:48:29.06324322 +0000 UTC m=+3.033809917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:28.164330 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.164285    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:28.164509 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.164339    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:28.164509 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.164493    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:28.164648 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.164557    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret podName:9efdb610-3597-487e-91c0-b0e85263d225 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:29.164535078 +0000 UTC m=+3.135101778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret") pod "global-pull-secret-syncer-6b6sc" (UID: "9efdb610-3597-487e-91c0-b0e85263d225") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:28.164987 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.164969    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:48:28.165073 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.164992    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:48:28.165073 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.165006    2571 projected.go:196] Error preparing data for projected volume kube-api-access-2r5pw for pod openshift-network-diagnostics/network-check-target-n7th9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:28.165073 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.165052    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw podName:4b0a6d42-8655-44e9-b30b-301aca1d6115 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:29.165036774 +0000 UTC m=+3.135603478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r5pw" (UniqueName: "kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw") pod "network-check-target-n7th9" (UID: "4b0a6d42-8655-44e9-b30b-301aca1d6115") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:28.493150 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.492997    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:43:28 +0000 UTC" deadline="2028-03-29 15:31:27.439860191 +0000 UTC"
Sep 09 20:48:28.493150 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.493067    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13602h42m58.946797651s"
Sep 09 20:48:28.562863 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.561833    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:28.562863 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.561969    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:28.562863 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.562411    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:28.562863 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.562503    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:28.562863 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.562569    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:28.562863 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:28.562647    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:28.572589 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.572530    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal" event={"ID":"bed195b806fdc3af1a773cfd2f90a22f","Type":"ContainerStarted","Data":"70271572febf830cc5ef30b06036c7c520ab9aaa660b3b509aabb522ec60a36f"}
Sep 09 20:48:28.579099 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.579060    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr" event={"ID":"f1716ba5-d90b-4532-ad98-9a0865c4a39c","Type":"ContainerStarted","Data":"05a308badb9964c20b1b05290015a42507f5277bb31201d9d87bf699d9852cf9"}
Sep 09 20:48:28.598260 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.598122    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bw5dh" event={"ID":"84d06660-1020-464f-93f3-084afba5ab9d","Type":"ContainerStarted","Data":"f47ca923ac6943d33617e7745cb24a06ffdaa2ac6631b861c7ca005893a2d2dd"}
Sep 09 20:48:28.605848 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.605745    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nlzcr" event={"ID":"76649ca0-30fa-40ea-8854-2ddab01b1692","Type":"ContainerStarted","Data":"7fec4053e110ba0703a83e544d24c61654f66e391cab38761a02e84499048b2a"}
Sep 09 20:48:28.615594 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.615378    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 20:48:28.620334 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.620222    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerStarted","Data":"16fda0f4868105f9ff2c0c0cca6693890e664d6e92d648a4c254bbde809123b3"}
Sep 09 20:48:28.630455 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.630416    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"4dda968ee73fb452037a1383b97d291f85a71d9262a155c600501fc6936febd9"}
Sep 09 20:48:28.647317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.647275    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal" event={"ID":"37da337e78d8a9155f29cc132c49e73a","Type":"ContainerStarted","Data":"920dd7f005204d90f30c4fbe657504693e147b338dc19381948eab6b168cb7b7"}
Sep 09 20:48:28.671266 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.671163    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6lkd2" event={"ID":"124b5fb2-9d37-4818-94f7-07af39967174","Type":"ContainerStarted","Data":"01a7f10ad149eba16ddb7055f27303310126fdf470ab1d783ecc077f1df8a33a"}
Sep 09 20:48:28.686328 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.686276    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5zn5j" event={"ID":"f8fc68ca-d113-4505-9b99-7581588403eb","Type":"ContainerStarted","Data":"3d7252f76f828253d4d9fd28f5442e9adf3802cad3fb0eafcf303eefbd18408c"}
Sep 09 20:48:28.697539 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.697496    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kff9p" event={"ID":"ea05bcec-cefd-42ad-968c-81c631ca98a3","Type":"ContainerStarted","Data":"fcebc201753b9d2d37cdc3fbc8ed11735ce66368a76c13f4baaf4f41092b5c71"}
Sep 09 20:48:28.704833 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:28.704791    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rp6l9" event={"ID":"237b6000-010e-4262-9ed4-9cfd02d1df7d","Type":"ContainerStarted","Data":"93d9fb641d7ad07b9918999bf6aa87757ffb374c565fd84e82653c8e4d39b5fe"}
Sep 09 20:48:29.071916 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:29.071875    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:29.072154 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:29.072036    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:29.072154 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:29.072109    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:48:31.07208814 +0000 UTC m=+5.042654839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:29.174196 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:29.173249    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:29.174196 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:29.173326    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:29.174196 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:29.173492    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:29.174196 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:29.173558    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret podName:9efdb610-3597-487e-91c0-b0e85263d225 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:31.173540025 +0000 UTC m=+5.144106723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret") pod "global-pull-secret-syncer-6b6sc" (UID: "9efdb610-3597-487e-91c0-b0e85263d225") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:29.174196 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:29.174051    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:48:29.174196 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:29.174078    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:48:29.174196 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:29.174091    2571 projected.go:196] Error preparing data for projected volume kube-api-access-2r5pw for pod openshift-network-diagnostics/network-check-target-n7th9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:29.174196 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:29.174140    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw podName:4b0a6d42-8655-44e9-b30b-301aca1d6115 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:31.174125052 +0000 UTC m=+5.144691754 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r5pw" (UniqueName: "kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw") pod "network-check-target-n7th9" (UID: "4b0a6d42-8655-44e9-b30b-301aca1d6115") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:29.494342 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:29.494196    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 20:43:28 +0000 UTC" deadline="2028-03-05 12:26:27.6150643 +0000 UTC"
Sep 09 20:48:29.494342 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:29.494245    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13023h37m58.120825179s"
Sep 09 20:48:30.562622 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:30.562517    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:30.563162 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:30.562663    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:30.563162 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:30.563072    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:30.563294 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:30.563208    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:30.563294 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:30.563262    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:30.563404 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:30.563338    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:31.097773 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:31.097725    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:31.097966 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:31.097908    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:31.098034 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:31.097977    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:48:35.097956888 +0000 UTC m=+9.068523600 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:31.198749 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:31.198689    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:31.198749 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:31.198749    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:31.199041 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:31.198894    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:31.199041 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:31.198960    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret podName:9efdb610-3597-487e-91c0-b0e85263d225 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:35.19894153 +0000 UTC m=+9.169508230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret") pod "global-pull-secret-syncer-6b6sc" (UID: "9efdb610-3597-487e-91c0-b0e85263d225") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:31.199214 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:31.199049    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:48:31.199214 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:31.199069    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:48:31.199214 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:31.199083    2571 projected.go:196] Error preparing data for projected volume kube-api-access-2r5pw for pod openshift-network-diagnostics/network-check-target-n7th9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:31.199214 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:31.199118    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw podName:4b0a6d42-8655-44e9-b30b-301aca1d6115 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:35.199106631 +0000 UTC m=+9.169673334 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r5pw" (UniqueName: "kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw") pod "network-check-target-n7th9" (UID: "4b0a6d42-8655-44e9-b30b-301aca1d6115") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:32.562238 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:32.561341    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:32.562238 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:32.561418    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:32.562238 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:32.561437    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:32.562238 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:32.561545    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:32.562238 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:32.562070    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:32.562238 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:32.562199    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:34.560732 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:34.560275    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:34.560732 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:34.560275    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:34.560732 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:34.560421    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:34.560732 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:34.560549    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:34.560732 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:34.560547    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:34.560732 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:34.560684    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:35.134241 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:35.134195    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:35.134433 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:35.134333    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:35.134433 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:35.134420    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:48:43.134398234 +0000 UTC m=+17.104964945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:35.235136 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:35.235079    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:35.235136 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:35.235142    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:35.235397 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:35.235281    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:35.235397 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:35.235335    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret podName:9efdb610-3597-487e-91c0-b0e85263d225 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:43.235319654 +0000 UTC m=+17.205886351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret") pod "global-pull-secret-syncer-6b6sc" (UID: "9efdb610-3597-487e-91c0-b0e85263d225") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:35.235514 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:35.235501    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:48:35.235557 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:35.235522    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:48:35.235557 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:35.235536    2571 projected.go:196] Error preparing data for projected volume kube-api-access-2r5pw for pod openshift-network-diagnostics/network-check-target-n7th9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:35.235620 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:35.235567    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw podName:4b0a6d42-8655-44e9-b30b-301aca1d6115 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:43.235558557 +0000 UTC m=+17.206125268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r5pw" (UniqueName: "kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw") pod "network-check-target-n7th9" (UID: "4b0a6d42-8655-44e9-b30b-301aca1d6115") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:36.561226 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:36.561150    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:36.561724 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:36.561288    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:36.561724 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:36.561325    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:36.561724 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:36.561188    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:36.561724 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:36.561475    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:36.561724 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:36.561552    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:38.559328 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:38.559281    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:38.559839 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:38.559281    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:38.559839 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:38.559454    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:38.559839 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:38.559286    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:38.559839 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:38.559496    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:38.559839 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:38.559612    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:40.559942 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:40.559900    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:40.559942 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:40.559924    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:40.559942 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:40.559897    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:40.560673 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:40.560057    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:40.560673 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:40.560149    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:40.560673 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:40.560220    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:42.560000 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:42.559968    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:42.560460 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:42.559968    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:42.560460 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:42.560119    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:42.560626 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:42.560589    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:42.566770 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:42.559970    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:42.566909 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:42.566880    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:43.204217 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:43.204184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:43.204432 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:43.204312    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:43.204432 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:43.204370    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:48:59.204355956 +0000 UTC m=+33.174922650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:43.304939 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:43.304900    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:43.304939 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:43.304944    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:43.305217 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:43.305095    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:48:43.305217 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:43.305122    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:48:43.305217 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:43.305119    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:43.305217 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:43.305135    2571 projected.go:196] Error preparing data for projected volume kube-api-access-2r5pw for pod openshift-network-diagnostics/network-check-target-n7th9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:43.305217 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:43.305189    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret podName:9efdb610-3597-487e-91c0-b0e85263d225 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:59.305167796 +0000 UTC m=+33.275734505 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret") pod "global-pull-secret-syncer-6b6sc" (UID: "9efdb610-3597-487e-91c0-b0e85263d225") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:43.305217 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:43.305207    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw podName:4b0a6d42-8655-44e9-b30b-301aca1d6115 nodeName:}" failed. No retries permitted until 2026-09-09 20:48:59.305199187 +0000 UTC m=+33.275765883 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r5pw" (UniqueName: "kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw") pod "network-check-target-n7th9" (UID: "4b0a6d42-8655-44e9-b30b-301aca1d6115") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:44.560341 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:44.560250    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:44.560894 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:44.560383    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:44.560894 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:44.560397    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:44.560894 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:44.560392    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:44.560894 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:44.560498    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:44.560894 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:44.560581    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:46.560449 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:46.560413    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:46.560872 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:46.560530    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:46.560872 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:46.560561    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:46.561265 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:46.561228    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:46.561377 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:46.561306    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:46.561741 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:46.561460    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:48.559620 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.559416    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:48.560167 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.559414    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:48.560167 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:48.559763    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:48.560167 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:48.559825    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:48.560167 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.559442    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:48.560167 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:48.559896    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:48.753276 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.753069    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6lkd2" event={"ID":"124b5fb2-9d37-4818-94f7-07af39967174","Type":"ContainerStarted","Data":"69f4ab5c32e18f24f29753ca6503ae850f8ec1409215b0f9dcd97ab41317cd45"}
Sep 09 20:48:48.754644 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.754621    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kff9p" event={"ID":"ea05bcec-cefd-42ad-968c-81c631ca98a3","Type":"ContainerStarted","Data":"ad053420b9512208c9012f7f0bdb50fec59401b8556c7bbe3102f2c26d7756cb"}
Sep 09 20:48:48.755968 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.755943    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rp6l9" event={"ID":"237b6000-010e-4262-9ed4-9cfd02d1df7d","Type":"ContainerStarted","Data":"22e3b7a4811b4f659b1cb4f052cf8a2ebf8bdb6d6ccbca35ca6582fffcaf3800"}
Sep 09 20:48:48.757887 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.757863    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal" event={"ID":"bed195b806fdc3af1a773cfd2f90a22f","Type":"ContainerStarted","Data":"e063a530e9ff59cc26c051204df56eff537e6eb90beba93c52f51c6f0d45ca5d"}
Sep 09 20:48:48.759225 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.759207    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr" event={"ID":"f1716ba5-d90b-4532-ad98-9a0865c4a39c","Type":"ContainerStarted","Data":"55b6ccdefa70583059c582decd53d0519f3b4ea5e3370d5e2192da9c054e9dc5"}
Sep 09 20:48:48.760404 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.760383    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bw5dh" event={"ID":"84d06660-1020-464f-93f3-084afba5ab9d","Type":"ContainerStarted","Data":"3b473e316ea5f6a8949e691c8407d31801429b470c686bbfc5782e2793b93f83"}
Sep 09 20:48:48.761547 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.761526    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nlzcr" event={"ID":"76649ca0-30fa-40ea-8854-2ddab01b1692","Type":"ContainerStarted","Data":"6fd3e35118a64fa2f76e5bf06260d39b84b1a21fc1465e934b9576c5ede3fa25"}
Sep 09 20:48:48.763933 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.763581    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerStarted","Data":"fe542433662cec46dd2ef584ec8f6e108c0f37c39d459c8a8857bc3f32827013"}
Sep 09 20:48:48.767808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.767775    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"6b8392f6181683c7ece91f426392a8700b31681c355ff46e26ac5ccb16d57c0e"}
Sep 09 20:48:48.767910 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.767846    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"432baf01546874e940ffc130dd19d1f34725924e896c9659212f74979e54fcc8"}
Sep 09 20:48:48.767910 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.767894    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"3efeed9512c7fcd0a3acdaf610db595b7c829728b30cef8fcccd5e56cb47bb97"}
Sep 09 20:48:48.768011 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.767939    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"fe9279f36a8f83ab23eb73621c0ce5e04fb369421ee4e98947a49bd610869766"}
Sep 09 20:48:48.770135 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.770110    2571 generic.go:356] "Generic (PLEG): container finished" podID="37da337e78d8a9155f29cc132c49e73a" containerID="d160e3a0af5c66bab46659340abc478e8b446a362f42e88c575c50de1d0628ba" exitCode=0
Sep 09 20:48:48.770221 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.770147    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal" event={"ID":"37da337e78d8a9155f29cc132c49e73a","Type":"ContainerDied","Data":"d160e3a0af5c66bab46659340abc478e8b446a362f42e88c575c50de1d0628ba"}
Sep 09 20:48:48.799253 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.799208    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nlzcr" podStartSLOduration=1.6177113589999998 podStartE2EDuration="21.799192866s" totalImagesPullingTime="20.181481507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:48.799054323 +0000 UTC m=+22.769621039" watchObservedRunningTime="2026-09-09 20:48:48.799192866 +0000 UTC m=+22.769759582"
Sep 09 20:48:48.816115 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.815997    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bw5dh" podStartSLOduration=1.663362003 podStartE2EDuration="21.815984548s" totalImagesPullingTime="20.152622545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:48.815685704 +0000 UTC m=+22.786252420" watchObservedRunningTime="2026-09-09 20:48:48.815984548 +0000 UTC m=+22.786551264"
Sep 09 20:48:48.827413 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.827369    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-13.ec2.internal" podStartSLOduration=20.827354568 podStartE2EDuration="20.827354568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:48.82723999 +0000 UTC m=+22.797806707" watchObservedRunningTime="2026-09-09 20:48:48.827354568 +0000 UTC m=+22.797921275"
Sep 09 20:48:48.877827 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.877785    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rp6l9" podStartSLOduration=1.607401767 podStartE2EDuration="21.877769955s" totalImagesPullingTime="20.270368188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:48.877417054 +0000 UTC m=+22.847983770" watchObservedRunningTime="2026-09-09 20:48:48.877769955 +0000 UTC m=+22.848336671"
Sep 09 20:48:48.900302 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:48.899032    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kff9p" podStartSLOduration=1.326831214 podStartE2EDuration="21.899008724s" totalImagesPullingTime="20.57217751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:48.898005097 +0000 UTC m=+22.868571815" watchObservedRunningTime="2026-09-09 20:48:48.899008724 +0000 UTC m=+22.869575442"
Sep 09 20:48:49.773494 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.773414    2571 generic.go:356] "Generic (PLEG): container finished" podID="3f86e245-9338-4c00-ae1a-129411488355" containerID="fe542433662cec46dd2ef584ec8f6e108c0f37c39d459c8a8857bc3f32827013" exitCode=0
Sep 09 20:48:49.774304 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.773497    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerDied","Data":"fe542433662cec46dd2ef584ec8f6e108c0f37c39d459c8a8857bc3f32827013"}
Sep 09 20:48:49.776695 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.776669    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"b90451da8d03c99c11cb2e6bd92b4246b01b58be936eaee371e1062c8b6b3d1b"}
Sep 09 20:48:49.776803 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.776745    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"2c87b2c014c2f62f3b4541c7a945f3a87725a8e94320827978a6ec4fe42b7b89"}
Sep 09 20:48:49.778584 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.778557    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal" event={"ID":"37da337e78d8a9155f29cc132c49e73a","Type":"ContainerStarted","Data":"21515421eac2ad2e680aa8fd5bfcdd689ab0a238413c9db1aa9ff9346eebc024"}
Sep 09 20:48:49.780049 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.780024    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5zn5j" event={"ID":"f8fc68ca-d113-4505-9b99-7581588403eb","Type":"ContainerStarted","Data":"cd28ee3b7155f0b85ffadab5bc59feab3171972a7e43966953da3dee8929e57d"}
Sep 09 20:48:49.794653 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.794604    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6lkd2" podStartSLOduration=2.581195137 podStartE2EDuration="22.794583786s" totalImagesPullingTime="20.213388649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:48.920232104 +0000 UTC m=+22.890798821" watchObservedRunningTime="2026-09-09 20:48:49.794583786 +0000 UTC m=+23.765150504"
Sep 09 20:48:49.821006 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.820959    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-13.ec2.internal" podStartSLOduration=21.82094285 podStartE2EDuration="21.82094285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:49.807119898 +0000 UTC m=+23.777686617" watchObservedRunningTime="2026-09-09 20:48:49.82094285 +0000 UTC m=+23.791509567"
Sep 09 20:48:49.821169 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:49.821138    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5zn5j" podStartSLOduration=6.404336198 podStartE2EDuration="22.821132561s" totalImagesPullingTime="16.416796363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:49.820874353 +0000 UTC m=+23.791441070" watchObservedRunningTime="2026-09-09 20:48:49.821132561 +0000 UTC m=+23.791699277"
Sep 09 20:48:50.051323 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:50.051299    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 20:48:50.516163 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:50.516060    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T20:48:50.051317015Z","UUID":"4fae8a4e-e2db-4066-8e65-7d6f27cd51b6","Handler":null,"Name":"","Endpoint":""}
Sep 09 20:48:50.517436 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:50.517419    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 20:48:50.517540 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:50.517444    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 20:48:50.560121 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:50.560090    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:50.560121 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:50.560109    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:50.560362 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:50.560211    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:50.560362 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:50.560297    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:50.560362 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:50.560360    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:50.560508 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:50.560435    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:50.784620 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:50.784353    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr" event={"ID":"f1716ba5-d90b-4532-ad98-9a0865c4a39c","Type":"ContainerStarted","Data":"99024c4f1dfe9e3bb97e5a78b81749af36c374cfbd7bb5fcc435e6e49115ad97"}
Sep 09 20:48:51.575392 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:51.575357    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:51.576068 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:51.576044    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:51.789925 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:51.789886    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"4e9318c93b946b4d5baa63db29f62801a7de32c868257eca0aab3c5c3b735f31"}
Sep 09 20:48:51.792462 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:51.792411    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr" event={"ID":"f1716ba5-d90b-4532-ad98-9a0865c4a39c","Type":"ContainerStarted","Data":"1730d9a679043f31c68b55bb7683edcb27e8d89a0210822c206e7392d1cf1343"}
Sep 09 20:48:51.792693 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:51.792675    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:51.793180 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:51.793151    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6lkd2"
Sep 09 20:48:51.809834 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:51.809785    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdgrr" podStartSLOduration=2.117321436 podStartE2EDuration="24.809769966s" totalImagesPullingTime="22.69244853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:51.809343663 +0000 UTC m=+25.779910382" watchObservedRunningTime="2026-09-09 20:48:51.809769966 +0000 UTC m=+25.780336683"
Sep 09 20:48:52.559489 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:52.559455    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:52.559489 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:52.559489    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:52.559698 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:52.559455    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:52.559698 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:52.559556    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:52.559698 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:52.559654    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:52.559823 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:52.559739    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:53.803936 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:53.803899    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" event={"ID":"99e6a99b-0004-4de5-86c5-cd1c5a747f75","Type":"ContainerStarted","Data":"2b16e726ec031f1b999aa73c00fc319ffce21fdd29635a35ff8262e5f1e482d3"}
Sep 09 20:48:53.832418 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:53.832359    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2" podStartSLOduration=6.211686588 podStartE2EDuration="26.832344879s" totalImagesPullingTime="20.620658291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:48:53.831744181 +0000 UTC m=+27.802310897" watchObservedRunningTime="2026-09-09 20:48:53.832344879 +0000 UTC m=+27.802911595"
Sep 09 20:48:54.559913 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:54.559733    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:54.560092 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:54.559732    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:54.560092 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:54.559997    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:54.560183 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:54.559801    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:54.560183 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:54.560139    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:54.560258 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:54.560189    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:54.806615 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:54.806580    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:54.807039 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:54.806694    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:54.807039 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:54.806747    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:54.825975 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:54.825898    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:54.826127 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:54.826097    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:48:55.203902 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:55.203873    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6b6sc"]
Sep 09 20:48:55.204047 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:55.203975    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:55.204108 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:55.204062    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:55.207111 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:55.207071    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n7th9"]
Sep 09 20:48:55.207240 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:55.207178    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:55.207325 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:55.207277    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:55.208121 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:55.208100    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jffdc"]
Sep 09 20:48:55.208207 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:55.208183    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:55.208297 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:55.208278    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:55.810045 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:55.809965    2571 generic.go:356] "Generic (PLEG): container finished" podID="3f86e245-9338-4c00-ae1a-129411488355" containerID="46203d4a8c5f6287ac4cd2637a783ad3e536191c9ca16feea54721c0f6c807e2" exitCode=0
Sep 09 20:48:55.810615 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:55.810055    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerDied","Data":"46203d4a8c5f6287ac4cd2637a783ad3e536191c9ca16feea54721c0f6c807e2"}
Sep 09 20:48:56.561021 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:56.560983    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:56.561181 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:56.561127    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:57.559980 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:57.559946    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:57.560538 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:57.559946    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:57.560538 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:57.560062    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:57.560538 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:57.560148    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:57.817824 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:57.817741    2571 generic.go:356] "Generic (PLEG): container finished" podID="3f86e245-9338-4c00-ae1a-129411488355" containerID="7c66c70857e1059ad8d01d59daa7de44771f437ee8539c3874ee7ef6ff831261" exitCode=0
Sep 09 20:48:57.817970 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:57.817859    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerDied","Data":"7c66c70857e1059ad8d01d59daa7de44771f437ee8539c3874ee7ef6ff831261"}
Sep 09 20:48:58.559605 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:58.559429    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:58.559767 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:58.559677    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:48:59.216803 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:59.216760    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:59.217284 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.216965    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:59.217284 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.217050    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:49:31.21702786 +0000 UTC m=+65.187594560 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 20:48:59.317910 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:59.317869    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:48:59.317910 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:59.317909    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:59.318227 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.318010    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:59.318227 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.318052    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 20:48:59.318227 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.318069    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret podName:9efdb610-3597-487e-91c0-b0e85263d225 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:31.31805341 +0000 UTC m=+65.288620108 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret") pod "global-pull-secret-syncer-6b6sc" (UID: "9efdb610-3597-487e-91c0-b0e85263d225") : object "kube-system"/"original-pull-secret" not registered
Sep 09 20:48:59.318227 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.318095    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 20:48:59.318227 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.318113    2571 projected.go:196] Error preparing data for projected volume kube-api-access-2r5pw for pod openshift-network-diagnostics/network-check-target-n7th9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:59.318227 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.318175    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw podName:4b0a6d42-8655-44e9-b30b-301aca1d6115 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:31.318156566 +0000 UTC m=+65.288723275 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2r5pw" (UniqueName: "kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw") pod "network-check-target-n7th9" (UID: "4b0a6d42-8655-44e9-b30b-301aca1d6115") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 20:48:59.559210 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:59.559172    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:48:59.559210 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:59.559210    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:48:59.559367 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.559287    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:48:59.559451 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:48:59.559413    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:48:59.824697 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:59.824612    2571 generic.go:356] "Generic (PLEG): container finished" podID="3f86e245-9338-4c00-ae1a-129411488355" containerID="a478f153e6e003801cd875de92a0792cde49a390fefa5d958274b2153478375c" exitCode=0
Sep 09 20:48:59.824697 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:48:59.824676    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerDied","Data":"a478f153e6e003801cd875de92a0792cde49a390fefa5d958274b2153478375c"}
Sep 09 20:49:00.559929 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:00.559834    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:49:00.560469 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:00.559950    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n7th9" podUID="4b0a6d42-8655-44e9-b30b-301aca1d6115"
Sep 09 20:49:01.559925 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:01.559835    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:49:01.560214 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:01.559844    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:49:01.560214 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:01.559964    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6b6sc" podUID="9efdb610-3597-487e-91c0-b0e85263d225"
Sep 09 20:49:01.560214 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:01.560066    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:49:02.306126 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.306097    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-13.ec2.internal" event="NodeReady"
Sep 09 20:49:02.306314 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.306249    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 20:49:02.342230 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.342195    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mbcnl"]
Sep 09 20:49:02.370613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.370516    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bb5b4df67-6qsn4"]
Sep 09 20:49:02.370613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.370664    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:02.374929 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.374905    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jf22v\""
Sep 09 20:49:02.375162 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.375144    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 09 20:49:02.375423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.375402    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 09 20:49:02.387625 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.387596    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mbcnl"]
Sep 09 20:49:02.387743 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.387637    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8nr68"]
Sep 09 20:49:02.387743 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.387722    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.390353 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.390329    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-smmqf\""
Sep 09 20:49:02.390570 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.390547    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 20:49:02.390766 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.390746    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 20:49:02.390953 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.390937    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 20:49:02.396517 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.396374    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 20:49:02.405606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.405583    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nrrnq"]
Sep 09 20:49:02.405758 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.405714    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.409980 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.409954    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 20:49:02.410299 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.410282    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5zq52\""
Sep 09 20:49:02.410546 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.410529    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 20:49:02.427845 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.427810    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bb5b4df67-6qsn4"]
Sep 09 20:49:02.427961 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.427856    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8nr68"]
Sep 09 20:49:02.427961 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.427879    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nrrnq"]
Sep 09 20:49:02.427961 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.427917    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nrrnq"
Sep 09 20:49:02.430523 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.430496    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 20:49:02.430643 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.430589    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b96lg\""
Sep 09 20:49:02.430772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.430755    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 20:49:02.431609 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.431591    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 20:49:02.440787 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.440767    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f8c0270-6125-42c2-a87b-f72f7483ed01-nginx-conf\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:02.440885 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.440806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:02.525863 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.525826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:02.541884 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.541839    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-image-registry-private-configuration\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.541884 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.541891    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b6dd829-83b3-4d95-9ee8-241bcdc87792-ca-trust-extracted\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.541909    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02494caa-a335-4dd7-b8b2-571a5555e37e-tmp-dir\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.541933    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1c6a852-7538-44c4-aa73-0b3faa8a9049-cert\") pod \"ingress-canary-nrrnq\" (UID: \"d1c6a852-7538-44c4-aa73-0b3faa8a9049\") " pod="openshift-ingress-canary/ingress-canary-nrrnq"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.541987    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542006    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-installation-pull-secrets\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542022    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/02494caa-a335-4dd7-b8b2-571a5555e37e-config-volume\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542045    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5dpn\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-kube-api-access-m5dpn\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542072    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-bound-sa-token\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542102    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjfx7\" (UniqueName: \"kubernetes.io/projected/02494caa-a335-4dd7-b8b2-571a5555e37e-kube-api-access-bjfx7\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.542116 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542118    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56bss\" (UniqueName: \"kubernetes.io/projected/d1c6a852-7538-44c4-aa73-0b3faa8a9049-kube-api-access-56bss\") pod \"ingress-canary-nrrnq\" (UID: \"d1c6a852-7538-44c4-aa73-0b3faa8a9049\") " pod="openshift-ingress-canary/ingress-canary-nrrnq"
Sep 09 20:49:02.542500 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f8c0270-6125-42c2-a87b-f72f7483ed01-nginx-conf\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:02.542500 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542244    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-certificates\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.542500 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542366    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.542500 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542414    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-trusted-ca\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.542500 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542467    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:02.542727 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:02.542594    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:49:02.542727 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:02.542668    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:03.042646407 +0000 UTC m=+37.013213103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:49:02.542891 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.542873    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4f8c0270-6125-42c2-a87b-f72f7483ed01-nginx-conf\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:02.560250 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.560170    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:49:02.563200 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.562898    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 20:49:02.563200 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.562902    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m7n9z\""
Sep 09 20:49:02.563200 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.562968    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 20:49:02.606012 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.605971    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:02.642967 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.642928    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-image-registry-private-configuration\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.642967 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.642972    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b6dd829-83b3-4d95-9ee8-241bcdc87792-ca-trust-extracted\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.642990    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02494caa-a335-4dd7-b8b2-571a5555e37e-tmp-dir\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643006    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1c6a852-7538-44c4-aa73-0b3faa8a9049-cert\") pod \"ingress-canary-nrrnq\" (UID: \"d1c6a852-7538-44c4-aa73-0b3faa8a9049\") " pod="openshift-ingress-canary/ingress-canary-nrrnq"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643034    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643050    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-installation-pull-secrets\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/02494caa-a335-4dd7-b8b2-571a5555e37e-config-volume\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643121    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5dpn\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-kube-api-access-m5dpn\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643140    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-bound-sa-token\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjfx7\" (UniqueName: \"kubernetes.io/projected/02494caa-a335-4dd7-b8b2-571a5555e37e-kube-api-access-bjfx7\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643172    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56bss\" (UniqueName: \"kubernetes.io/projected/d1c6a852-7538-44c4-aa73-0b3faa8a9049-kube-api-access-56bss\") pod \"ingress-canary-nrrnq\" (UID: \"d1c6a852-7538-44c4-aa73-0b3faa8a9049\") " pod="openshift-ingress-canary/ingress-canary-nrrnq"
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:02.643188    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:49:02.643243 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:02.643213    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:02.643284    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:03.143260687 +0000 UTC m=+37.113827399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643201    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-certificates\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643338    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643386    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-trusted-ca\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643410    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02494caa-a335-4dd7-b8b2-571a5555e37e-tmp-dir\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643468    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b6dd829-83b3-4d95-9ee8-241bcdc87792-ca-trust-extracted\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:02.643556    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:02.643609    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:49:03.143590788 +0000 UTC m=+37.114157499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:49:02.643717 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.643672    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-certificates\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.644261 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.644241    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-trusted-ca\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.644313 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.644291    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/02494caa-a335-4dd7-b8b2-571a5555e37e-config-volume\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.647919 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.647895    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-image-registry-private-configuration\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.648044 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.647938    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-installation-pull-secrets\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.648044 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.647955    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1c6a852-7538-44c4-aa73-0b3faa8a9049-cert\") pod \"ingress-canary-nrrnq\" (UID: \"d1c6a852-7538-44c4-aa73-0b3faa8a9049\") " pod="openshift-ingress-canary/ingress-canary-nrrnq"
Sep 09 20:49:02.652637 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.652607    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjfx7\" (UniqueName: \"kubernetes.io/projected/02494caa-a335-4dd7-b8b2-571a5555e37e-kube-api-access-bjfx7\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:02.652795 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.652721    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5dpn\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-kube-api-access-m5dpn\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.652945 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.652921    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-bound-sa-token\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:02.653114 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.653050    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56bss\" (UniqueName: \"kubernetes.io/projected/d1c6a852-7538-44c4-aa73-0b3faa8a9049-kube-api-access-56bss\") pod \"ingress-canary-nrrnq\" (UID: \"d1c6a852-7538-44c4-aa73-0b3faa8a9049\") " pod="openshift-ingress-canary/ingress-canary-nrrnq"
Sep 09 20:49:02.736235 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:02.736187    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nrrnq"
Sep 09 20:49:03.041403 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.041196    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nrrnq"]
Sep 09 20:49:03.047267 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.047237    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:03.047432 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:03.047408    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:49:03.047536 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:03.047486    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:04.047467861 +0000 UTC m=+38.018034556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:49:03.147888 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.147792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:03.147888 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.147858    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:03.148143 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:03.147943    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:49:03.148143 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:03.147953    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:49:03.148143 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:03.147966    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:49:03.148143 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:03.148029    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:04.148007336 +0000 UTC m=+38.118574045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:49:03.148143 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:03.148044    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:49:04.148036543 +0000 UTC m=+38.118603238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:49:03.559764 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.559728    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:49:03.559962 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.559731    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:49:03.561994 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.561969    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 20:49:03.562352 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.562093    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 20:49:03.562352 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:03.562240    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v6qsr\""
Sep 09 20:49:04.055635 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:04.055589    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:04.055859 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:04.055754    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:49:04.055859 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:04.055849    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:06.055828656 +0000 UTC m=+40.026395368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:49:04.156328 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:04.156287    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:04.156485 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:04.156394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:04.156485 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:04.156470    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:49:04.156584 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:04.156493    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:49:04.156584 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:04.156509    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:49:04.156584 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:04.156545    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:49:06.156529864 +0000 UTC m=+40.127096560 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:49:04.156584 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:04.156564    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:06.156554524 +0000 UTC m=+40.127121218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:49:04.504990 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:49:04.504960    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c6a852_7538_44c4_aa73_0b3faa8a9049.slice/crio-76b70b30fc69c6e02a4cc5dd457a9ee945e4d064b55d30ebafc3862bbc1f2785 WatchSource:0}: Error finding container 76b70b30fc69c6e02a4cc5dd457a9ee945e4d064b55d30ebafc3862bbc1f2785: Status 404 returned error can't find the container with id 76b70b30fc69c6e02a4cc5dd457a9ee945e4d064b55d30ebafc3862bbc1f2785
Sep 09 20:49:04.841257 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:04.841222    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerStarted","Data":"4cfea673957e905ebb4785393bea2a4f4980c54e4a356197de2dd23ee79232f5"}
Sep 09 20:49:04.842547 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:04.842523    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nrrnq" event={"ID":"d1c6a852-7538-44c4-aa73-0b3faa8a9049","Type":"ContainerStarted","Data":"76b70b30fc69c6e02a4cc5dd457a9ee945e4d064b55d30ebafc3862bbc1f2785"}
Sep 09 20:49:05.848405 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:05.848362    2571 generic.go:356] "Generic (PLEG): container finished" podID="3f86e245-9338-4c00-ae1a-129411488355" containerID="4cfea673957e905ebb4785393bea2a4f4980c54e4a356197de2dd23ee79232f5" exitCode=0
Sep 09 20:49:05.848854 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:05.848419    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerDied","Data":"4cfea673957e905ebb4785393bea2a4f4980c54e4a356197de2dd23ee79232f5"}
Sep 09 20:49:05.965139 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:05.965072    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:06.052897 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:06.052868    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:06.072899 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:06.072853    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:06.073114 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:06.073001    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:49:06.073114 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:06.073099    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:10.073060894 +0000 UTC m=+44.043627606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:49:06.173418 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:06.173326    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:06.173583 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:06.173459    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:06.173583 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:06.173519    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:49:06.173583 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:06.173572    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:49:06.173748 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:06.173586    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:49:06.173748 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:06.173593    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:49:10.173571484 +0000 UTC m=+44.144138178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:49:06.173748 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:06.173636    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:10.173620045 +0000 UTC m=+44.144186765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:49:06.176224 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:06.176158    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:06.276829 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:06.276794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:06.855027 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:06.854993    2571 generic.go:356] "Generic (PLEG): container finished" podID="3f86e245-9338-4c00-ae1a-129411488355" containerID="d01af370390fc73ccf0a8194d5732a07e8c00e62a2252245e003c823f9ef34be" exitCode=0
Sep 09 20:49:06.855502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:06.855054    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerDied","Data":"d01af370390fc73ccf0a8194d5732a07e8c00e62a2252245e003c823f9ef34be"}
Sep 09 20:49:07.184840 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:07.184753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:07.260372 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:07.260341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:07.861243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:07.861169    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7j67z" event={"ID":"3f86e245-9338-4c00-ae1a-129411488355","Type":"ContainerStarted","Data":"9c303154020a6a62bf6b8b20035e053b00c5e1334e0d3190dfc22355c8170813"}
Sep 09 20:49:07.880150 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:07.880091    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7j67z" podStartSLOduration=7.53398653 podStartE2EDuration="40.880056544s" totalImagesPullingTime="33.346070014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:49:07.87931533 +0000 UTC m=+41.849882048" watchObservedRunningTime="2026-09-09 20:49:07.880056544 +0000 UTC m=+41.850623296"
Sep 09 20:49:08.004318 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:08.004287    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:08.082623 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:08.082578    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:08.144022 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:08.143945    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:08.865444 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:08.865404    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nrrnq" event={"ID":"d1c6a852-7538-44c4-aa73-0b3faa8a9049","Type":"ContainerStarted","Data":"c189312954ca5f048d9ffe6c09bb3833df0cab838ff936c7b1f65d8b56c51d6e"}
Sep 09 20:49:08.882721 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:08.882661    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nrrnq" podStartSLOduration=3.500039987 podStartE2EDuration="6.882644556s" totalImagesPullingTime="3.382604569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:49:08.881994056 +0000 UTC m=+42.852560775" watchObservedRunningTime="2026-09-09 20:49:08.882644556 +0000 UTC m=+42.853211327"
Sep 09 20:49:09.377063 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:09.377030    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:09.477804 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:09.477767    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:10.107703 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:10.107664    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:10.108100 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:10.107805    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:49:10.108100 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:10.107865    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:18.107850626 +0000 UTC m=+52.078417321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:49:10.208849 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:10.208811    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:10.208982 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:10.208915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:10.209039 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:10.208978    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:49:10.209039 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:10.209003    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:49:10.209039 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:10.209014    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:49:10.209158 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:10.209051    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:49:18.209032182 +0000 UTC m=+52.179598891 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:49:10.209158 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:10.209066    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:18.209060526 +0000 UTC m=+52.179627221 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:49:11.338456 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:11.338422    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:11.437996 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:11.437968    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:15.674255 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:15.674215    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:15.727575 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:15.727540    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:17.865446 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:17.865411    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:17.914858 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:17.914824    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:18.172652 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:18.172554    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:18.172798 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:18.172714    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:49:18.172798 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:18.172781    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:34.172765415 +0000 UTC m=+68.143332109 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:49:18.273521 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:18.273470    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:18.273693 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:18.273540    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:18.273693 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:18.273623    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:49:18.273693 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:18.273643    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:49:18.273803 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:18.273686    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:49:18.273803 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:18.273697    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:49:34.273683267 +0000 UTC m=+68.244249962 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:49:18.273803 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:18.273753    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:49:34.273740804 +0000 UTC m=+68.244307498 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:49:22.445827 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:22.445791    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:22.500626 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:22.500593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:24.262237 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:24.262155    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:24.329574 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:24.329549    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:26.832769 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:26.832744    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wsth2"
Sep 09 20:49:26.985473 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:26.985435    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:27.041486 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:27.041454    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:31.262018 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.261976    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:49:31.264520 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.264501    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 20:49:31.273201 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:31.273164    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 20:49:31.273346 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:31.273293    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:50:35.273271429 +0000 UTC m=+129.243838125 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : secret "metrics-daemon-secret" not found
Sep 09 20:49:31.362940 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.362905    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:49:31.362940 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.362944    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:49:31.365502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.365482    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 20:49:31.365625 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.365486    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 20:49:31.375612 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.375594    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 20:49:31.376752 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.376734    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9efdb610-3597-487e-91c0-b0e85263d225-original-pull-secret\") pod \"global-pull-secret-syncer-6b6sc\" (UID: \"9efdb610-3597-487e-91c0-b0e85263d225\") " pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:49:31.386484 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.386457    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r5pw\" (UniqueName: \"kubernetes.io/projected/4b0a6d42-8655-44e9-b30b-301aca1d6115-kube-api-access-2r5pw\") pod \"network-check-target-n7th9\" (UID: \"4b0a6d42-8655-44e9-b30b-301aca1d6115\") " pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:49:31.470803 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.470760    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6b6sc"
Sep 09 20:49:31.673466 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.673426    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m7n9z\""
Sep 09 20:49:31.682090 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.682051    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:49:31.736327 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.736299    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6b6sc"]
Sep 09 20:49:31.739651 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:49:31.739619    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efdb610_3597_487e_91c0_b0e85263d225.slice/crio-558f8632bbe4a1d9c8edfe8f5cee3ac7f5888f39d046297b367ee91f76a4c53a WatchSource:0}: Error finding container 558f8632bbe4a1d9c8edfe8f5cee3ac7f5888f39d046297b367ee91f76a4c53a: Status 404 returned error can't find the container with id 558f8632bbe4a1d9c8edfe8f5cee3ac7f5888f39d046297b367ee91f76a4c53a
Sep 09 20:49:31.926013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.925925    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6b6sc" event={"ID":"9efdb610-3597-487e-91c0-b0e85263d225","Type":"ContainerStarted","Data":"558f8632bbe4a1d9c8edfe8f5cee3ac7f5888f39d046297b367ee91f76a4c53a"}
Sep 09 20:49:31.946109 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:31.946065    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n7th9"]
Sep 09 20:49:31.948733 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:49:31.948708    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0a6d42_8655_44e9_b30b_301aca1d6115.slice/crio-613e31ad60d39aa3a34161187abac14cc42e145c71eb2eb8f87ee6c5eba9ef21 WatchSource:0}: Error finding container 613e31ad60d39aa3a34161187abac14cc42e145c71eb2eb8f87ee6c5eba9ef21: Status 404 returned error can't find the container with id 613e31ad60d39aa3a34161187abac14cc42e145c71eb2eb8f87ee6c5eba9ef21
Sep 09 20:49:32.363030 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:32.362992    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:32.425962 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:32.425925    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:32.930651 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:32.930610    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n7th9" event={"ID":"4b0a6d42-8655-44e9-b30b-301aca1d6115","Type":"ContainerStarted","Data":"613e31ad60d39aa3a34161187abac14cc42e145c71eb2eb8f87ee6c5eba9ef21"}
Sep 09 20:49:34.184979 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:34.184928    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:49:34.185447 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:34.185111    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:49:34.185447 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:34.185184    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:06.185168303 +0000 UTC m=+100.155734998 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:49:34.285716 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:34.285677    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:49:34.285968 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:34.285737    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:49:34.285968 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:34.285858    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:49:34.285968 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:34.285888    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:49:34.285968 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:34.285956    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:50:06.285935095 +0000 UTC m=+100.256501793 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:49:34.285968 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:34.285866    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:49:34.286272 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:49:34.286032    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:50:06.286010935 +0000 UTC m=+100.256577660 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:49:36.943982 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:36.943943    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n7th9" event={"ID":"4b0a6d42-8655-44e9-b30b-301aca1d6115","Type":"ContainerStarted","Data":"cfc2e23a42f01460163ba12f88a86e7fe6b49387bec3974f9bcd41467116816a"}
Sep 09 20:49:36.944432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:36.944044    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:49:36.945764 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:36.945738    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6b6sc" event={"ID":"9efdb610-3597-487e-91c0-b0e85263d225","Type":"ContainerStarted","Data":"890f8b7b992ff812b7e8185d2b2b069cc5e30897506fd2283ae36fa53eb63463"}
Sep 09 20:49:36.961280 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:36.961232    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n7th9" podStartSLOduration=65.155961374 podStartE2EDuration="1m9.961217692s" totalImagesPullingTime="4.805256318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:49:36.959486318 +0000 UTC m=+70.930053034" watchObservedRunningTime="2026-09-09 20:49:36.961217692 +0000 UTC m=+70.931784400"
Sep 09 20:49:36.974065 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:36.973987    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6b6sc" podStartSLOduration=64.954038329 podStartE2EDuration="1m9.973970475s" totalImagesPullingTime="5.019932146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:49:36.973909813 +0000 UTC m=+70.944476530" watchObservedRunningTime="2026-09-09 20:49:36.973970475 +0000 UTC m=+70.944537194"
Sep 09 20:49:45.245594 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:45.245555    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:45.304381 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:45.304351    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:49:45.362565 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:49:45.362532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:06.212527 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:50:06.212466    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:50:06.212926 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:06.212632    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:50:06.212926 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:06.212727    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:51:10.212710051 +0000 UTC m=+164.183276747 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:50:06.313412 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:50:06.313359    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:50:06.313539 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:50:06.313470    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:50:06.313539 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:06.313525    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:50:06.313607 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:06.313573    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:50:06.313607 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:06.313585    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:50:06.313670 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:06.313612    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:51:10.313590967 +0000 UTC m=+164.284157684 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:50:06.313670 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:06.313630    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:51:10.313620287 +0000 UTC m=+164.284187002 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:50:07.950224 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:50:07.950197    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n7th9"
Sep 09 20:50:11.607638 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:50:11.607602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:11.658966 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:50:11.658934    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:50:35.327766 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:50:35.327722    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:50:35.328309 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:35.327853    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 20:50:35.328309 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:50:35.327917    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:52:37.32790246 +0000 UTC m=+251.298469154 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : secret "metrics-daemon-secret" not found
Sep 09 20:51:05.383977 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:05.383937    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl" podUID="4f8c0270-6125-42c2-a87b-f72f7483ed01"
Sep 09 20:51:05.400139 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:05.400096    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" podUID="9b6dd829-83b3-4d95-9ee8-241bcdc87792"
Sep 09 20:51:05.420280 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:05.420247    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8nr68" podUID="02494caa-a335-4dd7-b8b2-571a5555e37e"
Sep 09 20:51:06.154672 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:51:06.154635    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8nr68"
Sep 09 20:51:06.154672 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:51:06.154666    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:51:06.154936 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:51:06.154781    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:51:06.577579 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:06.577542    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:51:10.280874 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:51:10.280836    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:51:10.281334 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:10.280986    2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 20:51:10.281334 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:10.281056    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert podName:4f8c0270-6125-42c2-a87b-f72f7483ed01 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:12.28103953 +0000 UTC m=+286.251606225 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-mbcnl" (UID: "4f8c0270-6125-42c2-a87b-f72f7483ed01") : secret "networking-console-plugin-cert" not found
Sep 09 20:51:10.381990 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:51:10.381940    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:51:10.382132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:51:10.382004    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:51:10.382132 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:10.382122    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 20:51:10.382199 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:10.382129    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 20:51:10.382199 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:10.382149    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bb5b4df67-6qsn4: secret "image-registry-tls" not found
Sep 09 20:51:10.382199 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:10.382197    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls podName:02494caa-a335-4dd7-b8b2-571a5555e37e nodeName:}" failed. No retries permitted until 2026-09-09 20:53:12.382182265 +0000 UTC m=+286.352748964 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls") pod "dns-default-8nr68" (UID: "02494caa-a335-4dd7-b8b2-571a5555e37e") : secret "dns-default-metrics-tls" not found
Sep 09 20:51:10.382298 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:51:10.382210    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls podName:9b6dd829-83b3-4d95-9ee8-241bcdc87792 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:12.382205023 +0000 UTC m=+286.352771717 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls") pod "image-registry-7bb5b4df67-6qsn4" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792") : secret "image-registry-tls" not found
Sep 09 20:51:21.559960 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:51:21.559885    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:52:23.617601 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:23.617565    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:37.394329 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:37.394276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:52:37.394791 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:37.394417    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 09 20:52:37.394791 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:37.394481    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs podName:d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc nodeName:}" failed. No retries permitted until 2026-09-09 20:54:39.394463395 +0000 UTC m=+373.365030091 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs") pod "network-metrics-daemon-jffdc" (UID: "d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc") : secret "metrics-daemon-secret" not found
Sep 09 20:52:55.038916 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.038836    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb"]
Sep 09 20:52:55.041627 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.041609    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb"
Sep 09 20:52:55.044128 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.044105    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 09 20:52:55.045017 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.045000    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:52:55.045121 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.045006    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kps89\""
Sep 09 20:52:55.046555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.046537    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"]
Sep 09 20:52:55.049258 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.049240    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-jlprg"]
Sep 09 20:52:55.049345 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.049335    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.052573 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.052553    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 09 20:52:55.052681 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.052599    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.053278 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.053220    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:52:55.053375 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.053289    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-r2wg4\""
Sep 09 20:52:55.053375 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.053346    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 09 20:52:55.053498 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.053480    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 09 20:52:55.053679 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.053663    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb"]
Sep 09 20:52:55.055242 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.055130    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 09 20:52:55.055242 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.055193    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 20:52:55.055661 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.055480    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 09 20:52:55.055661 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.055489    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-xwsd2\""
Sep 09 20:52:55.055811 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.055696    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 20:52:55.059923 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.059903    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 09 20:52:55.060585 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.060569    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"]
Sep 09 20:52:55.066849 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.066829    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-jlprg"]
Sep 09 20:52:55.122895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.122859    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/54399796-4e0e-4fb9-803a-7d5c20f66b4d-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.122895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.122892    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbvgg\" (UniqueName: \"kubernetes.io/projected/54399796-4e0e-4fb9-803a-7d5c20f66b4d-kube-api-access-tbvgg\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.123140 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.122910    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-tmp\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.123140 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.122924    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-snapshots\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.123140 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.122957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.123140 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.122982    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5hz2\" (UniqueName: \"kubernetes.io/projected/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-kube-api-access-k5hz2\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.123140 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.123010    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j855s\" (UniqueName: \"kubernetes.io/projected/1ae94f88-c80d-4afa-941c-e43185426d12-kube-api-access-j855s\") pod \"volume-data-source-validator-695c895bd6-zx6nb\" (UID: \"1ae94f88-c80d-4afa-941c-e43185426d12\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb"
Sep 09 20:52:55.123140 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.123056    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/54399796-4e0e-4fb9-803a-7d5c20f66b4d-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.123140 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.123113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.123347 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.123153    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-serving-cert\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.131936 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.130411    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-grx2v"]
Sep 09 20:52:55.134232 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.134204    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-grx2v"
Sep 09 20:52:55.136822 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.136796    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-97lwk\""
Sep 09 20:52:55.141169 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.141139    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"]
Sep 09 20:52:55.144277 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.144262    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:55.146929 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.146911    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:52:55.148236 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.148204    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 09 20:52:55.148338 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.148211    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 09 20:52:55.148338 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.148255    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hhcvp\""
Sep 09 20:52:55.148922 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.148903    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-grx2v"]
Sep 09 20:52:55.165190 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.165166    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"]
Sep 09 20:52:55.224293 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224262    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5hz2\" (UniqueName: \"kubernetes.io/projected/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-kube-api-access-k5hz2\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.224293 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224298    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/54399796-4e0e-4fb9-803a-7d5c20f66b4d-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.224555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-tmp\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.224555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224333    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.224555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224350    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-snapshots\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.224555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6gdm\" (UniqueName: \"kubernetes.io/projected/3679545c-af32-4d5f-8ab1-ef3efce5cc20-kube-api-access-c6gdm\") pod \"network-check-source-685456655f-grx2v\" (UID: \"3679545c-af32-4d5f-8ab1-ef3efce5cc20\") " pod="openshift-network-diagnostics/network-check-source-685456655f-grx2v"
Sep 09 20:52:55.224555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224408    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/54399796-4e0e-4fb9-803a-7d5c20f66b4d-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.224555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224489    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-serving-cert\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.224555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224550    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbvgg\" (UniqueName: \"kubernetes.io/projected/54399796-4e0e-4fb9-803a-7d5c20f66b4d-kube-api-access-tbvgg\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.224892 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224596    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz4bj\" (UniqueName: \"kubernetes.io/projected/eabeccde-f485-4732-aa8c-43472ce3b915-kube-api-access-vz4bj\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:55.224892 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224638    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.224892 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224673    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j855s\" (UniqueName: \"kubernetes.io/projected/1ae94f88-c80d-4afa-941c-e43185426d12-kube-api-access-j855s\") pod \"volume-data-source-validator-695c895bd6-zx6nb\" (UID: \"1ae94f88-c80d-4afa-941c-e43185426d12\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb"
Sep 09 20:52:55.224892 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224693    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:55.224892 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224762    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-tmp\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.225152 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.224986    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/54399796-4e0e-4fb9-803a-7d5c20f66b4d-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.225152 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.225054    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-snapshots\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.225231 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.225216    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.226004 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.225985    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.227207 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.227178    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-serving-cert\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.227326 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.227309    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/54399796-4e0e-4fb9-803a-7d5c20f66b4d-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.230036 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.230014    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"]
Sep 09 20:52:55.234023 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.234004    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.236266 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.236247    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:52:55.236390 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.236278    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 09 20:52:55.236472 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.236413    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 09 20:52:55.236472 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.236420    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-95ks9\""
Sep 09 20:52:55.236567 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.236504    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 09 20:52:55.243563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.243535    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5hz2\" (UniqueName: \"kubernetes.io/projected/caaa9f3d-3587-4330-ac7f-e3982c2c09a4-kube-api-access-k5hz2\") pod \"insights-operator-76bcd6ccf5-jlprg\" (UID: \"caaa9f3d-3587-4330-ac7f-e3982c2c09a4\") " pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.243685 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.243652    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbvgg\" (UniqueName: \"kubernetes.io/projected/54399796-4e0e-4fb9-803a-7d5c20f66b4d-kube-api-access-tbvgg\") pod \"kube-storage-version-migrator-operator-8d7dd4675-dkb8z\" (UID: \"54399796-4e0e-4fb9-803a-7d5c20f66b4d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.243685 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.243667    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"]
Sep 09 20:52:55.243772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.243686    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j855s\" (UniqueName: \"kubernetes.io/projected/1ae94f88-c80d-4afa-941c-e43185426d12-kube-api-access-j855s\") pod \"volume-data-source-validator-695c895bd6-zx6nb\" (UID: \"1ae94f88-c80d-4afa-941c-e43185426d12\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb"
Sep 09 20:52:55.322226 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.322132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:55.325019 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.324996    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d153405f-bb69-4223-8a81-f3751898f01c-config\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.325114 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.325030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:55.325114 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.325062    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6gdm\" (UniqueName: \"kubernetes.io/projected/3679545c-af32-4d5f-8ab1-ef3efce5cc20-kube-api-access-c6gdm\") pod \"network-check-source-685456655f-grx2v\" (UID: \"3679545c-af32-4d5f-8ab1-ef3efce5cc20\") " pod="openshift-network-diagnostics/network-check-source-685456655f-grx2v"
Sep 09 20:52:55.325221 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.325136    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d153405f-bb69-4223-8a81-f3751898f01c-serving-cert\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.325221 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:55.325167    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:52:55.325344 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:55.325221    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls podName:eabeccde-f485-4732-aa8c-43472ce3b915 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:55.825204109 +0000 UTC m=+269.795770810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-lj9ml" (UID: "eabeccde-f485-4732-aa8c-43472ce3b915") : secret "samples-operator-tls" not found
Sep 09 20:52:55.325344 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.325271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz4bj\" (UniqueName: \"kubernetes.io/projected/eabeccde-f485-4732-aa8c-43472ce3b915-kube-api-access-vz4bj\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:55.325344 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.325318    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ktml\" (UniqueName: \"kubernetes.io/projected/d153405f-bb69-4223-8a81-f3751898f01c-kube-api-access-2ktml\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.334263 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.334235    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6gdm\" (UniqueName: \"kubernetes.io/projected/3679545c-af32-4d5f-8ab1-ef3efce5cc20-kube-api-access-c6gdm\") pod \"network-check-source-685456655f-grx2v\" (UID: \"3679545c-af32-4d5f-8ab1-ef3efce5cc20\") " pod="openshift-network-diagnostics/network-check-source-685456655f-grx2v"
Sep 09 20:52:55.334393 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.334352    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz4bj\" (UniqueName: \"kubernetes.io/projected/eabeccde-f485-4732-aa8c-43472ce3b915-kube-api-access-vz4bj\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:55.351168 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.351140    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb"
Sep 09 20:52:55.361904 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.361831    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"
Sep 09 20:52:55.367757 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.367733    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg"
Sep 09 20:52:55.426492 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.426376    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d153405f-bb69-4223-8a81-f3751898f01c-serving-cert\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.426677 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.426651    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ktml\" (UniqueName: \"kubernetes.io/projected/d153405f-bb69-4223-8a81-f3751898f01c-kube-api-access-2ktml\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.426768 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.426710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d153405f-bb69-4223-8a81-f3751898f01c-config\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.427355 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.427292    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d153405f-bb69-4223-8a81-f3751898f01c-config\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.429741 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.429483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d153405f-bb69-4223-8a81-f3751898f01c-serving-cert\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.437801 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.437754    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ktml\" (UniqueName: \"kubernetes.io/projected/d153405f-bb69-4223-8a81-f3751898f01c-kube-api-access-2ktml\") pod \"service-ca-operator-859f94c4b-psg9n\" (UID: \"d153405f-bb69-4223-8a81-f3751898f01c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.443258 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.443230    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-grx2v"
Sep 09 20:52:55.543691 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.543650    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"
Sep 09 20:52:55.625893 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.625860    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb"]
Sep 09 20:52:55.629179 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:52:55.629153    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae94f88_c80d_4afa_941c_e43185426d12.slice/crio-473d7fbac7ca874e04a6367e26fc2a6874d6bf5987db12a9331f58e86994465e WatchSource:0}: Error finding container 473d7fbac7ca874e04a6367e26fc2a6874d6bf5987db12a9331f58e86994465e: Status 404 returned error can't find the container with id 473d7fbac7ca874e04a6367e26fc2a6874d6bf5987db12a9331f58e86994465e
Sep 09 20:52:55.637724 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.637707    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z"]
Sep 09 20:52:55.639968 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:52:55.639944    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54399796_4e0e_4fb9_803a_7d5c20f66b4d.slice/crio-3a1f7000a4556ec5f6233187a292c60a1a11ba3ff85c06c86f8f401570aca1c7 WatchSource:0}: Error finding container 3a1f7000a4556ec5f6233187a292c60a1a11ba3ff85c06c86f8f401570aca1c7: Status 404 returned error can't find the container with id 3a1f7000a4556ec5f6233187a292c60a1a11ba3ff85c06c86f8f401570aca1c7
Sep 09 20:52:55.661778 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.661748    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-jlprg"]
Sep 09 20:52:55.664656 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:52:55.664631    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaaa9f3d_3587_4330_ac7f_e3982c2c09a4.slice/crio-694405f28b99665cd7f57bf05e47ce8a681c8178b603aea6de5c77d00cc04754 WatchSource:0}: Error finding container 694405f28b99665cd7f57bf05e47ce8a681c8178b603aea6de5c77d00cc04754: Status 404 returned error can't find the container with id 694405f28b99665cd7f57bf05e47ce8a681c8178b603aea6de5c77d00cc04754
Sep 09 20:52:55.709912 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.709885    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-grx2v"]
Sep 09 20:52:55.712448 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:52:55.712425    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3679545c_af32_4d5f_8ab1_ef3efce5cc20.slice/crio-8353fa1d2861fb60fb15f157f6d7eec26146d4bb3c7edfcafee5e271544c6eb6 WatchSource:0}: Error finding container 8353fa1d2861fb60fb15f157f6d7eec26146d4bb3c7edfcafee5e271544c6eb6: Status 404 returned error can't find the container with id 8353fa1d2861fb60fb15f157f6d7eec26146d4bb3c7edfcafee5e271544c6eb6
Sep 09 20:52:55.811048 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.811018    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n"]
Sep 09 20:52:55.813859 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:52:55.813829    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd153405f_bb69_4223_8a81_f3751898f01c.slice/crio-ed16c1d7477502f8b22dda209a0575062fa9e3e4678c94d5f4f970e584f5ddf0 WatchSource:0}: Error finding container ed16c1d7477502f8b22dda209a0575062fa9e3e4678c94d5f4f970e584f5ddf0: Status 404 returned error can't find the container with id ed16c1d7477502f8b22dda209a0575062fa9e3e4678c94d5f4f970e584f5ddf0
Sep 09 20:52:55.830570 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:55.830549    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:55.830723 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:55.830707    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:52:55.830782 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:55.830761    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls podName:eabeccde-f485-4732-aa8c-43472ce3b915 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:56.830746991 +0000 UTC m=+270.801313686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-lj9ml" (UID: "eabeccde-f485-4732-aa8c-43472ce3b915") : secret "samples-operator-tls" not found
Sep 09 20:52:56.419738 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.419695    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z" event={"ID":"54399796-4e0e-4fb9-803a-7d5c20f66b4d","Type":"ContainerStarted","Data":"3a1f7000a4556ec5f6233187a292c60a1a11ba3ff85c06c86f8f401570aca1c7"}
Sep 09 20:52:56.421510 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.421478    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-grx2v" event={"ID":"3679545c-af32-4d5f-8ab1-ef3efce5cc20","Type":"ContainerStarted","Data":"33498c7dd8ff1e2bce4de5ad29ed3da1dc6cd69de8d01042b9ce4f387b94676e"}
Sep 09 20:52:56.421645 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.421530    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-grx2v" event={"ID":"3679545c-af32-4d5f-8ab1-ef3efce5cc20","Type":"ContainerStarted","Data":"8353fa1d2861fb60fb15f157f6d7eec26146d4bb3c7edfcafee5e271544c6eb6"}
Sep 09 20:52:56.422853 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.422827    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb" event={"ID":"1ae94f88-c80d-4afa-941c-e43185426d12","Type":"ContainerStarted","Data":"473d7fbac7ca874e04a6367e26fc2a6874d6bf5987db12a9331f58e86994465e"}
Sep 09 20:52:56.424137 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.424113    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n" event={"ID":"d153405f-bb69-4223-8a81-f3751898f01c","Type":"ContainerStarted","Data":"ed16c1d7477502f8b22dda209a0575062fa9e3e4678c94d5f4f970e584f5ddf0"}
Sep 09 20:52:56.425482 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.425457    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg" event={"ID":"caaa9f3d-3587-4330-ac7f-e3982c2c09a4","Type":"ContainerStarted","Data":"694405f28b99665cd7f57bf05e47ce8a681c8178b603aea6de5c77d00cc04754"}
Sep 09 20:52:56.437046 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.436991    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-grx2v" podStartSLOduration=1.436970877 podStartE2EDuration="1.436970877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:56.436058195 +0000 UTC m=+270.406624912" watchObservedRunningTime="2026-09-09 20:52:56.436970877 +0000 UTC m=+270.407537596"
Sep 09 20:52:56.836501 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.836464    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:56.836688 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:56.836656    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:52:56.836753 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:56.836726    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls podName:eabeccde-f485-4732-aa8c-43472ce3b915 nodeName:}" failed. No retries permitted until 2026-09-09 20:52:58.836710867 +0000 UTC m=+272.807277562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-lj9ml" (UID: "eabeccde-f485-4732-aa8c-43472ce3b915") : secret "samples-operator-tls" not found
Sep 09 20:52:56.994773 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:56.994662    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:58.856243 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:58.856193    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:52:58.856694 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:58.856354    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:52:58.856694 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:52:58.856438    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls podName:eabeccde-f485-4732-aa8c-43472ce3b915 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:02.8564171 +0000 UTC m=+276.826983809 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-lj9ml" (UID: "eabeccde-f485-4732-aa8c-43472ce3b915") : secret "samples-operator-tls" not found
Sep 09 20:52:59.438604 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.438566    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n" event={"ID":"d153405f-bb69-4223-8a81-f3751898f01c","Type":"ContainerStarted","Data":"060476f5fb6ec7ad6bd85fac704e2273908636e49c59fb353f92ce4390e5fa5d"}
Sep 09 20:52:59.440286 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.440252    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg" event={"ID":"caaa9f3d-3587-4330-ac7f-e3982c2c09a4","Type":"ContainerStarted","Data":"af38401729e2b81f6ad82c772ed3e37a0a1d38cff51f679396f0f4eed2e432cd"}
Sep 09 20:52:59.442163 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.442139    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z" event={"ID":"54399796-4e0e-4fb9-803a-7d5c20f66b4d","Type":"ContainerStarted","Data":"94c85fc34e3b74ed8bfa16c59ab29d8a935c84639d6e1567b4a1114aa79023b1"}
Sep 09 20:52:59.443804 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.443780    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb" event={"ID":"1ae94f88-c80d-4afa-941c-e43185426d12","Type":"ContainerStarted","Data":"98d3a2dd5d484ea969c67db55478355f95f1303cd5dc60beb1013eb773f7cb3a"}
Sep 09 20:52:59.457413 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.457366    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n" podStartSLOduration=1.313717965 podStartE2EDuration="4.457352303s" totalImagesPullingTime="3.143634338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:59.455835559 +0000 UTC m=+273.426402277" watchObservedRunningTime="2026-09-09 20:52:59.457352303 +0000 UTC m=+273.427919014"
Sep 09 20:52:59.496098 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.495720    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-zx6nb" podStartSLOduration=1.172555411 podStartE2EDuration="4.495698305s" totalImagesPullingTime="3.323142894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:59.474384794 +0000 UTC m=+273.444951512" watchObservedRunningTime="2026-09-09 20:52:59.495698305 +0000 UTC m=+273.466265024"
Sep 09 20:52:59.496578 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.496540    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z" podStartSLOduration=1.183798815 podStartE2EDuration="4.496525641s" totalImagesPullingTime="3.312726826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:59.495239936 +0000 UTC m=+273.465806653" watchObservedRunningTime="2026-09-09 20:52:59.496525641 +0000 UTC m=+273.467092363"
Sep 09 20:52:59.520276 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.520225    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg" podStartSLOduration=1.230540806 podStartE2EDuration="4.520206301s" totalImagesPullingTime="3.289665495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:52:59.51789788 +0000 UTC m=+273.488464597" watchObservedRunningTime="2026-09-09 20:52:59.520206301 +0000 UTC m=+273.490773093"
Sep 09 20:52:59.595150 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.595108    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:52:59.673983 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:52:59.673950    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:00.241899 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.241855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:00.371893 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.371853    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-58h2w"]
Sep 09 20:53:00.376509 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.376475    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.379326 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.379297    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ngfdf\""
Sep 09 20:53:00.379673 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.379647    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 20:53:00.379942 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.379920    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 20:53:00.380660 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.380633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:00.432375 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.432344    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-58h2w"]
Sep 09 20:53:00.469336 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.469304    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98rfk\" (UniqueName: \"kubernetes.io/projected/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-api-access-98rfk\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.469530 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.469459    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e389e63-e95d-4cac-bef6-7bd051dfd17d-crio-socket\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.469787 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.469765    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e389e63-e95d-4cac-bef6-7bd051dfd17d-data-volume\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.469929 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.469896    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.470627 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.470594    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.571426 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.571334    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.571426 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.571416    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.571606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.571438    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98rfk\" (UniqueName: \"kubernetes.io/projected/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-api-access-98rfk\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.571606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.571470    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e389e63-e95d-4cac-bef6-7bd051dfd17d-crio-socket\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.571606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.571493    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e389e63-e95d-4cac-bef6-7bd051dfd17d-data-volume\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.571606 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:00.571518    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:00.571606 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:00.571552    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:53:00.571606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.571596    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0e389e63-e95d-4cac-bef6-7bd051dfd17d-crio-socket\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.571829 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:00.571607    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:01.071587749 +0000 UTC m=+275.042154448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:00.571829 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:00.571648    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:01.07163306 +0000 UTC m=+275.042199758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : configmap "kube-rbac-proxy" not found
Sep 09 20:53:00.571829 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.571777    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0e389e63-e95d-4cac-bef6-7bd051dfd17d-data-volume\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.580366 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.580340    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98rfk\" (UniqueName: \"kubernetes.io/projected/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-api-access-98rfk\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:00.882008 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.881917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:00.981239 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:00.981206    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:01.074600 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:01.074567    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:01.074783 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:01.074642    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:01.074783 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:01.074708    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:53:01.074783 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:01.074749    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:01.074924 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:01.074761    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:02.074747406 +0000 UTC m=+276.045314101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : configmap "kube-rbac-proxy" not found
Sep 09 20:53:01.074924 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:01.074829    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:02.07480794 +0000 UTC m=+276.045374648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:01.083644 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:01.083606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:01.272560 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:01.272518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:01.374391 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:01.374356    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:02.084033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.083998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:02.084236 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.084067    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:02.084236 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:02.084163    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:53:02.084236 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:02.084185    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:02.084333 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:02.084242    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:04.084220177 +0000 UTC m=+278.054786872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : configmap "kube-rbac-proxy" not found
Sep 09 20:53:02.084333 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:02.084257    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:04.084251195 +0000 UTC m=+278.054817889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:02.819798 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.819758    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8lh96"]
Sep 09 20:53:02.822984 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.822965    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.825864 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.825845    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 09 20:53:02.826421 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.826395    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-snrzd\""
Sep 09 20:53:02.826421 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.826411    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 09 20:53:02.826696 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.826675    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 09 20:53:02.826781 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.826755    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 09 20:53:02.833469 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.833445    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8lh96"]
Sep 09 20:53:02.883319 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.883289    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:02.889284 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.889257    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-signing-cabundle\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.889411 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.889307    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh7mb\" (UniqueName: \"kubernetes.io/projected/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-kube-api-access-lh7mb\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.889453 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.889425    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:53:02.889492 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.889453    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-signing-key\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.889559 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:02.889546    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 20:53:02.889613 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:02.889605    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls podName:eabeccde-f485-4732-aa8c-43472ce3b915 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:10.889591718 +0000 UTC m=+284.860158413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-lj9ml" (UID: "eabeccde-f485-4732-aa8c-43472ce3b915") : secret "samples-operator-tls" not found
Sep 09 20:53:02.989984 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.989949    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh7mb\" (UniqueName: \"kubernetes.io/projected/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-kube-api-access-lh7mb\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.990171 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.990054    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-signing-key\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.990171 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.990096    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-signing-cabundle\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.990689 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.990668    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-signing-cabundle\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.992550 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.992529    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-signing-key\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:02.998740 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.998702    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:02.999114 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:02.999097    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh7mb\" (UniqueName: \"kubernetes.io/projected/f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6-kube-api-access-lh7mb\") pod \"service-ca-75d99c484-8lh96\" (UID: \"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6\") " pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:03.132197 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:03.132100    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-8lh96"
Sep 09 20:53:03.343766 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:03.343740    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nlzcr_76649ca0-30fa-40ea-8854-2ddab01b1692/dns-node-resolver/0.log"
Sep 09 20:53:03.412856 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:03.412778    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-8lh96"]
Sep 09 20:53:03.415613 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:03.415575    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f559b7_0f99_4cd7_8fbc_74949cb9f3f6.slice/crio-81f2c63139300e4537e4b9d448a6f1458c1a3633269fe7946ac0aba3a959b491 WatchSource:0}: Error finding container 81f2c63139300e4537e4b9d448a6f1458c1a3633269fe7946ac0aba3a959b491: Status 404 returned error can't find the container with id 81f2c63139300e4537e4b9d448a6f1458c1a3633269fe7946ac0aba3a959b491
Sep 09 20:53:03.457001 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:03.456966    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-8lh96" event={"ID":"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6","Type":"ContainerStarted","Data":"81f2c63139300e4537e4b9d448a6f1458c1a3633269fe7946ac0aba3a959b491"}
Sep 09 20:53:04.098311 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.098275    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:04.098692 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.098360    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:04.098692 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:04.098402    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:53:04.098692 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:04.098449    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:04.098692 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:04.098468    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:08.098452568 +0000 UTC m=+282.069019263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : configmap "kube-rbac-proxy" not found
Sep 09 20:53:04.098692 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:04.098490    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:08.098478308 +0000 UTC m=+282.069045029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:04.111999 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.111966    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:04.315628 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.315587    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:04.343512 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.343487    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rp6l9_237b6000-010e-4262-9ed4-9cfd02d1df7d/node-ca/0.log"
Sep 09 20:53:04.462980 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.462943    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-8lh96" event={"ID":"f6f559b7-0f99-4cd7-8fbc-74949cb9f3f6","Type":"ContainerStarted","Data":"0451c37e0b16c55d81e8352df4962a651c1c126c2c02d6cb22ce0dcf649caf2b"}
Sep 09 20:53:04.529720 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.529655    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-8lh96" podStartSLOduration=2.529634234 podStartE2EDuration="2.529634234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:04.526800025 +0000 UTC m=+278.497366743" watchObservedRunningTime="2026-09-09 20:53:04.529634234 +0000 UTC m=+278.500200952"
Sep 09 20:53:04.579828 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.579781    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:04.702845 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.702773    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:04.814938 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.814858    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:04.943455 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:04.943424    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nrrnq_d1c6a852-7538-44c4-aa73-0b3faa8a9049/serve-healthcheck-canary/0.log"
Sep 09 20:53:05.399238 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:05.399204    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:05.521793 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:05.521757    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:05.743865 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:05.743821    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-dkb8z_54399796-4e0e-4fb9-803a-7d5c20f66b4d/kube-storage-version-migrator-operator/0.log"
Sep 09 20:53:05.802775 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:05.802737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:06.228183 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:06.228132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:06.610873 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:06.610780    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:06.802194 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:06.802144    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:07.201762 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:07.201722    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:07.621238 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:07.621150    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:08.019203 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:08.019166    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:08.133767 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:08.133725    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:08.133928 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:08.133811    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:08.133928 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:08.133853    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 20:53:08.133928 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:08.133902    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:08.133928 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:08.133922    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:16.13390583 +0000 UTC m=+290.104472542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : configmap "kube-rbac-proxy" not found
Sep 09 20:53:08.134108 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:08.133958    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls podName:0e389e63-e95d-4cac-bef6-7bd051dfd17d nodeName:}" failed. No retries permitted until 2026-09-09 20:53:16.133945421 +0000 UTC m=+290.104512140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-58h2w" (UID: "0e389e63-e95d-4cac-bef6-7bd051dfd17d") : secret "insights-runtime-extractor-tls" not found
Sep 09 20:53:08.209732 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:08.209698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:08.807262 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:08.807231    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:08.911927 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:08.911891    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:09.107678 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:09.107589    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:09.155606 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:09.155571    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl" podUID="4f8c0270-6125-42c2-a87b-f72f7483ed01"
Sep 09 20:53:09.155606 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:09.155571    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-8nr68" podUID="02494caa-a335-4dd7-b8b2-571a5555e37e"
Sep 09 20:53:09.155784 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:09.155571    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" podUID="9b6dd829-83b3-4d95-9ee8-241bcdc87792"
Sep 09 20:53:09.483191 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:09.483156    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:53:09.483384 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:09.483164    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:53:09.483384 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:09.483164    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8nr68"
Sep 09 20:53:09.621356 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:09.621318    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.000609 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:10.000571    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.193968 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:10.193914    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.398142 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:10.398025    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.797010 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:10.796971    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.897611 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:10.897573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:10.960458 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:10.960413    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:53:10.963266 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:10.963235    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eabeccde-f485-4732-aa8c-43472ce3b915-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-lj9ml\" (UID: \"eabeccde-f485-4732-aa8c-43472ce3b915\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:53:11.053073 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:11.052994    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"
Sep 09 20:53:11.329050 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:11.328846    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml"]
Sep 09 20:53:11.400151 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:11.400122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:11.490970 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:11.490938    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml" event={"ID":"eabeccde-f485-4732-aa8c-43472ce3b915","Type":"ContainerStarted","Data":"cffabd4f36d1b8377845de235661b15121020b6a3cce81dffb66f13747063590"}
Sep 09 20:53:11.822350 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:11.822286    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:12.205141 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.201302    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:12.372257 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.372211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:53:12.375359 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.375329    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4f8c0270-6125-42c2-a87b-f72f7483ed01-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-mbcnl\" (UID: \"4f8c0270-6125-42c2-a87b-f72f7483ed01\") " pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:53:12.399224 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.399185    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:12.473255 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.473165    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:53:12.473429 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.473290    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:53:12.475606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.475572    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02494caa-a335-4dd7-b8b2-571a5555e37e-metrics-tls\") pod \"dns-default-8nr68\" (UID: \"02494caa-a335-4dd7-b8b2-571a5555e37e\") " pod="openshift-dns/dns-default-8nr68"
Sep 09 20:53:12.475879 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.475859    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"image-registry-7bb5b4df67-6qsn4\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") " pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:53:12.486756 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.486729    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jf22v\""
Sep 09 20:53:12.486873 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.486759    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5zq52\""
Sep 09 20:53:12.486873 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.486770    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-smmqf\""
Sep 09 20:53:12.493904 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.493856    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:53:12.493904 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.493891    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8nr68"
Sep 09 20:53:12.494064 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.493862    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl"
Sep 09 20:53:12.498971 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.498945    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:12.788026 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.787916    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-mbcnl"]
Sep 09 20:53:12.789041 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.788980    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bb5b4df67-6qsn4"]
Sep 09 20:53:12.790797 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.790690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8nr68"]
Sep 09 20:53:12.987289 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:12.987222    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:13.061677 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:13.061593    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6dd829_83b3_4d95_9ee8_241bcdc87792.slice/crio-28ebd1cfcdc31c5ec8c7966d6da9a264f3735d87590dcb9a342ecf94c5be8ae8 WatchSource:0}: Error finding container 28ebd1cfcdc31c5ec8c7966d6da9a264f3735d87590dcb9a342ecf94c5be8ae8: Status 404 returned error can't find the container with id 28ebd1cfcdc31c5ec8c7966d6da9a264f3735d87590dcb9a342ecf94c5be8ae8
Sep 09 20:53:13.062551 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:13.062529    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02494caa_a335_4dd7_b8b2_571a5555e37e.slice/crio-63df0513c276a73912493ddffdce9aa242365f4be9b058d8ccf93faf70bd4f58 WatchSource:0}: Error finding container 63df0513c276a73912493ddffdce9aa242365f4be9b058d8ccf93faf70bd4f58: Status 404 returned error can't find the container with id 63df0513c276a73912493ddffdce9aa242365f4be9b058d8ccf93faf70bd4f58
Sep 09 20:53:13.062996 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:13.062972    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f8c0270_6125_42c2_a87b_f72f7483ed01.slice/crio-52667e2628fcb150d89612a44ada3cb638e4637e0291c79e8d83faece5119bdd WatchSource:0}: Error finding container 52667e2628fcb150d89612a44ada3cb638e4637e0291c79e8d83faece5119bdd: Status 404 returned error can't find the container with id 52667e2628fcb150d89612a44ada3cb638e4637e0291c79e8d83faece5119bdd
Sep 09 20:53:13.408121 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.408013    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:13.499802 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.499758    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl" event={"ID":"4f8c0270-6125-42c2-a87b-f72f7483ed01","Type":"ContainerStarted","Data":"52667e2628fcb150d89612a44ada3cb638e4637e0291c79e8d83faece5119bdd"}
Sep 09 20:53:13.501696 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.501661    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" event={"ID":"9b6dd829-83b3-4d95-9ee8-241bcdc87792","Type":"ContainerStarted","Data":"a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01"}
Sep 09 20:53:13.501859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.501714    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" event={"ID":"9b6dd829-83b3-4d95-9ee8-241bcdc87792","Type":"ContainerStarted","Data":"28ebd1cfcdc31c5ec8c7966d6da9a264f3735d87590dcb9a342ecf94c5be8ae8"}
Sep 09 20:53:13.501859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.501772    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:53:13.503940 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.503914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml" event={"ID":"eabeccde-f485-4732-aa8c-43472ce3b915","Type":"ContainerStarted","Data":"5852ca6dd10317a23db20c19fc9e2f28fa061a47cd3d6dfbe43674f22f4c9b14"}
Sep 09 20:53:13.504095 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.503969    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml" event={"ID":"eabeccde-f485-4732-aa8c-43472ce3b915","Type":"ContainerStarted","Data":"43f9cd09fcb5059d006139d8d4d30eaf386f6f44e9c96a99beb31ae5f01b6e19"}
Sep 09 20:53:13.505203 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.505178    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8nr68" event={"ID":"02494caa-a335-4dd7-b8b2-571a5555e37e","Type":"ContainerStarted","Data":"63df0513c276a73912493ddffdce9aa242365f4be9b058d8ccf93faf70bd4f58"}
Sep 09 20:53:13.523001 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.522940    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" podStartSLOduration=285.522923246 podStartE2EDuration="4m45.522923246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:13.521606902 +0000 UTC m=+287.492173619" watchObservedRunningTime="2026-09-09 20:53:13.522923246 +0000 UTC m=+287.493489960"
Sep 09 20:53:13.540003 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:13.539947    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-lj9ml" podStartSLOduration=16.80238982 podStartE2EDuration="18.539930326s" totalImagesPullingTime="1.737540506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:13.539245768 +0000 UTC m=+287.509812495" watchObservedRunningTime="2026-09-09 20:53:13.539930326 +0000 UTC m=+287.510497042"
Sep 09 20:53:14.511381 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:14.511342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:14.611689 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:14.611645    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:14.800160 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:14.800018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:15.255718 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.255683    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:15.327707 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.327620    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:15.515042 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.515010    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8nr68" event={"ID":"02494caa-a335-4dd7-b8b2-571a5555e37e","Type":"ContainerStarted","Data":"c7226a9465142157e7fd38742bb0b5db9cbee36e0807d65c193349f8a31bfb44"}
Sep 09 20:53:15.515542 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.515062    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8nr68" event={"ID":"02494caa-a335-4dd7-b8b2-571a5555e37e","Type":"ContainerStarted","Data":"fdb38ea5a6734722d1b8ec4128f8e320711dfbbdb82f6d33161184cc145917f5"}
Sep 09 20:53:15.515542 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.515124    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8nr68"
Sep 09 20:53:15.516580 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.516560    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl" event={"ID":"4f8c0270-6125-42c2-a87b-f72f7483ed01","Type":"ContainerStarted","Data":"119be2b0519d0dbc909aec860b665db7aa03c28af1f633e1f8d1e32fafeffc7f"}
Sep 09 20:53:15.535542 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.535498    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8nr68" podStartSLOduration=251.633320907 podStartE2EDuration="4m13.535482639s" totalImagesPullingTime="1.902161732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:15.534443543 +0000 UTC m=+289.505010260" watchObservedRunningTime="2026-09-09 20:53:15.535482639 +0000 UTC m=+289.506049356"
Sep 09 20:53:15.556223 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.556163    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-mbcnl" podStartSLOduration=263.654631007 podStartE2EDuration="4m25.556147543s" totalImagesPullingTime="1.901516536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:15.555008493 +0000 UTC m=+289.525575211" watchObservedRunningTime="2026-09-09 20:53:15.556147543 +0000 UTC m=+289.526714291"
Sep 09 20:53:15.694611 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.694573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:15.752653 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:15.752623    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:16.211472 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:16.211433    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:16.211642 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:16.211501    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:16.212064 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:16.212045    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0e389e63-e95d-4cac-bef6-7bd051dfd17d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:16.213925 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:16.213908    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0e389e63-e95d-4cac-bef6-7bd051dfd17d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-58h2w\" (UID: \"0e389e63-e95d-4cac-bef6-7bd051dfd17d\") " pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:16.287159 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:16.287125    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-58h2w"
Sep 09 20:53:16.556545 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:16.556457    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-58h2w"]
Sep 09 20:53:16.559785 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:16.559761    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e389e63_e95d_4cac_bef6_7bd051dfd17d.slice/crio-ed96a6cd13b721fe6ebfb226c4bfd418b2ff14a10b17a09c503dc0484c94063a WatchSource:0}: Error finding container ed96a6cd13b721fe6ebfb226c4bfd418b2ff14a10b17a09c503dc0484c94063a: Status 404 returned error can't find the container with id ed96a6cd13b721fe6ebfb226c4bfd418b2ff14a10b17a09c503dc0484c94063a
Sep 09 20:53:17.524660 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:17.524624    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-58h2w" event={"ID":"0e389e63-e95d-4cac-bef6-7bd051dfd17d","Type":"ContainerStarted","Data":"379bcd842c59ca6dccae779210497bf961d04967c51d1eb51c92f8a310900f92"}
Sep 09 20:53:17.524805 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:17.524680    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-58h2w" event={"ID":"0e389e63-e95d-4cac-bef6-7bd051dfd17d","Type":"ContainerStarted","Data":"ed96a6cd13b721fe6ebfb226c4bfd418b2ff14a10b17a09c503dc0484c94063a"}
Sep 09 20:53:18.531270 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:18.531232    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-58h2w" event={"ID":"0e389e63-e95d-4cac-bef6-7bd051dfd17d","Type":"ContainerStarted","Data":"7b9689dd3ba8ef7c7f5301e9c3ce3266edfc915cfdf9930b45af8c4581617691"}
Sep 09 20:53:19.536509 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:19.536470    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-58h2w" event={"ID":"0e389e63-e95d-4cac-bef6-7bd051dfd17d","Type":"ContainerStarted","Data":"2b6a91d5202b811ea5346db34e5425129fe47af0385a6ce3013cb4056c6f6211"}
Sep 09 20:53:19.568222 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:19.568172    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-58h2w" podStartSLOduration=17.184737033 podStartE2EDuration="19.568155693s" totalImagesPullingTime="2.38341866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 20:53:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:19.567316738 +0000 UTC m=+293.537883451" watchObservedRunningTime="2026-09-09 20:53:19.568155693 +0000 UTC m=+293.538722410"
Sep 09 20:53:19.763340 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:19.763304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:19.865242 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:19.865153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:20.782344 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:20.782296    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:20.881345 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:20.881309    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:21.427025 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:21.426989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:21.626692 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:21.626653    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:21.683121 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:21.683016    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:22.239620 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:22.239580    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:22.307169 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:22.307130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:24.560779 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:24.560744    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-jffdc" podUID="d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc"
Sep 09 20:53:25.203381 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.203343    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-5n6ft"]
Sep 09 20:53:25.206555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.206535    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-5n6ft"
Sep 09 20:53:25.209151 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.209125    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 20:53:25.210032 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.210011    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mlddk\""
Sep 09 20:53:25.210032 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.210022    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 20:53:25.218564 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.218544    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-5n6ft"]
Sep 09 20:53:25.281268 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.281238    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4kxh\" (UniqueName: \"kubernetes.io/projected/270c8650-2237-4f09-ac0e-9d7c6b803e17-kube-api-access-h4kxh\") pod \"downloads-85d98c998f-5n6ft\" (UID: \"270c8650-2237-4f09-ac0e-9d7c6b803e17\") " pod="openshift-console/downloads-85d98c998f-5n6ft"
Sep 09 20:53:25.290878 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.290845    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:25.381613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.381578    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4kxh\" (UniqueName: \"kubernetes.io/projected/270c8650-2237-4f09-ac0e-9d7c6b803e17-kube-api-access-h4kxh\") pod \"downloads-85d98c998f-5n6ft\" (UID: \"270c8650-2237-4f09-ac0e-9d7c6b803e17\") " pod="openshift-console/downloads-85d98c998f-5n6ft"
Sep 09 20:53:25.388628 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.388593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:25.391237 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.391216    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4kxh\" (UniqueName: \"kubernetes.io/projected/270c8650-2237-4f09-ac0e-9d7c6b803e17-kube-api-access-h4kxh\") pod \"downloads-85d98c998f-5n6ft\" (UID: \"270c8650-2237-4f09-ac0e-9d7c6b803e17\") " pod="openshift-console/downloads-85d98c998f-5n6ft"
Sep 09 20:53:25.515561 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.515477    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-5n6ft"
Sep 09 20:53:25.522007 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.521986    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8nr68"
Sep 09 20:53:25.798121 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:25.798025    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-5n6ft"]
Sep 09 20:53:25.801734 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:25.801703    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod270c8650_2237_4f09_ac0e_9d7c6b803e17.slice/crio-366f1df80926261a56a1cfb546f5e5a4053ae531f5e9b723563fa67d2ce5d1bd WatchSource:0}: Error finding container 366f1df80926261a56a1cfb546f5e5a4053ae531f5e9b723563fa67d2ce5d1bd: Status 404 returned error can't find the container with id 366f1df80926261a56a1cfb546f5e5a4053ae531f5e9b723563fa67d2ce5d1bd
Sep 09 20:53:26.481459 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:26.481418    2571 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 20:53:26.565742 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:26.565583    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-5n6ft" event={"ID":"270c8650-2237-4f09-ac0e-9d7c6b803e17","Type":"ContainerStarted","Data":"366f1df80926261a56a1cfb546f5e5a4053ae531f5e9b723563fa67d2ce5d1bd"}
Sep 09 20:53:29.313838 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.313804    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5fb94fb5-xx9nj"]
Sep 09 20:53:29.317128 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.317104    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.319520 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.319492    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 20:53:29.319778 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.319752    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 20:53:29.320606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.320473    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f5srx\""
Sep 09 20:53:29.320606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.320510    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 20:53:29.320606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.320550    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 20:53:29.320606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.320567    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 20:53:29.328062 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.328040    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5fb94fb5-xx9nj"]
Sep 09 20:53:29.413859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.413823    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-oauth-serving-cert\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.413859 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.413860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-oauth-config\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.414127 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.413988    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-console-config\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.414127 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.414038    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-serving-cert\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.414248 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.414133    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-service-ca\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.414248 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.414166    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzqpp\" (UniqueName: \"kubernetes.io/projected/c40a6696-c9a7-4897-a094-7f30c33e6494-kube-api-access-pzqpp\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.506451 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.506414    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:29.515366 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.515333    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-console-config\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.515518 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.515390    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-serving-cert\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.515518 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.515446    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-service-ca\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.515518 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.515470    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzqpp\" (UniqueName: \"kubernetes.io/projected/c40a6696-c9a7-4897-a094-7f30c33e6494-kube-api-access-pzqpp\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.515518 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.515514    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-oauth-serving-cert\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.515727 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.515538    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-oauth-config\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.516270 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.516245    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-service-ca\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.516540 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.516290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-console-config\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.516540 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.516471    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-oauth-serving-cert\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.518827 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.518792    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-serving-cert\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.518921 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.518796    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-oauth-config\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.524474 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.524450    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzqpp\" (UniqueName: \"kubernetes.io/projected/c40a6696-c9a7-4897-a094-7f30c33e6494-kube-api-access-pzqpp\") pod \"console-5c5fb94fb5-xx9nj\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") " pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.607485 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.607391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:29.628808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.628765    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:29.677608 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.677573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:29.770452 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.770411    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:29.915433 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.915404    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5fb94fb5-xx9nj"]
Sep 09 20:53:29.917802 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:29.917771    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40a6696_c9a7_4897_a094_7f30c33e6494.slice/crio-497f585a0fc46ae21f3de124c60296492b309b18ca82fc8cb0d467e4f06b97ea WatchSource:0}: Error finding container 497f585a0fc46ae21f3de124c60296492b309b18ca82fc8cb0d467e4f06b97ea: Status 404 returned error can't find the container with id 497f585a0fc46ae21f3de124c60296492b309b18ca82fc8cb0d467e4f06b97ea
Sep 09 20:53:29.920022 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:29.920002    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:53:30.581295 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:30.581248    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5fb94fb5-xx9nj" event={"ID":"c40a6696-c9a7-4897-a094-7f30c33e6494","Type":"ContainerStarted","Data":"497f585a0fc46ae21f3de124c60296492b309b18ca82fc8cb0d467e4f06b97ea"}
Sep 09 20:53:32.499849 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:32.499783    2571 patch_prober.go:28] interesting pod/image-registry-7bb5b4df67-6qsn4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 09 20:53:32.500340 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:32.499854    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" podUID="9b6dd829-83b3-4d95-9ee8-241bcdc87792" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 20:53:32.694657 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:32.694583    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:32.781335 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:32.781230    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:33.061545 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:33.061460    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:33.596916 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:33.596881    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5fb94fb5-xx9nj" event={"ID":"c40a6696-c9a7-4897-a094-7f30c33e6494","Type":"ContainerStarted","Data":"8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3"}
Sep 09 20:53:33.616054 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:33.615987    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c5fb94fb5-xx9nj" podStartSLOduration=1.073292983 podStartE2EDuration="4.615963426s" totalImagesPullingTime="3.542670443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:33.615516066 +0000 UTC m=+307.586082793" watchObservedRunningTime="2026-09-09 20:53:33.615963426 +0000 UTC m=+307.586530146"
Sep 09 20:53:34.464253 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:34.464219    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:34.514987 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:34.514959    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:53:34.539044 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:34.538997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:34.832956 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:34.832876    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:34.916675 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:34.916632    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:35.381535 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:35.381497    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:36.544038 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:36.544002    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:36.607788 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:36.607742    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:37.653708 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.653670    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54bf487c68-rj7j6"]
Sep 09 20:53:37.658225 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.658202    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.668406 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.668377    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 20:53:37.669337 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.669313    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54bf487c68-rj7j6"]
Sep 09 20:53:37.690579 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.690550    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cqnr\" (UniqueName: \"kubernetes.io/projected/d58dcb4e-1478-4c22-9ba1-10bd526d6946-kube-api-access-4cqnr\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.690579 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.690581    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-trusted-ca-bundle\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.690779 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.690602    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-serving-cert\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.690779 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.690711    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-oauth-config\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.690779 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.690752    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-config\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.690872 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.690821    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-oauth-serving-cert\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.690872 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.690839    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-service-ca\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.731604 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.731570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:37.792190 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.792158    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-serving-cert\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.792376 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.792210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-oauth-config\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.792376 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.792243    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-config\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.792376 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.792291    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-oauth-serving-cert\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.792376 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.792316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-service-ca\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.792376 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.792359    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cqnr\" (UniqueName: \"kubernetes.io/projected/d58dcb4e-1478-4c22-9ba1-10bd526d6946-kube-api-access-4cqnr\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.792644 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.792383    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-trusted-ca-bundle\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.793929 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.793368    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-oauth-serving-cert\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.793929 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.793450    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-config\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.793929 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.793857    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-service-ca\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.795033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.794969    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-trusted-ca-bundle\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.796048 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.796023    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-serving-cert\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.796738 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.796706    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-oauth-config\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.816987 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.816959    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cqnr\" (UniqueName: \"kubernetes.io/projected/d58dcb4e-1478-4c22-9ba1-10bd526d6946-kube-api-access-4cqnr\") pod \"console-54bf487c68-rj7j6\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") " pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:37.862310 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.862274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:37.958570 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.958529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:37.969205 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:37.969166    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:38.072986 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.072943    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:38.078863 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.078832    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"]
Sep 09 20:53:38.082731 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.082710    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.086073 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.086045    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"]
Sep 09 20:53:38.088244 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.088209    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 09 20:53:38.088708 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.088688    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 09 20:53:38.089486 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.089470    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 20:53:38.090348 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.090332    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 20:53:38.091904 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.090865    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 20:53:38.091904 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.090858    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.091904 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.091362    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kd6dd\""
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095545    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095610    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095645    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095724    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87c8906a-3840-4dcf-afa7-3e85753b28a0-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095755    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl9zv\" (UniqueName: \"kubernetes.io/projected/87c8906a-3840-4dcf-afa7-3e85753b28a0-kube-api-access-kl9zv\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095847    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095939    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.095976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g2d4\" (UniqueName: \"kubernetes.io/projected/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-api-access-7g2d4\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.096195    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.096347    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.096496    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6qmcn\""
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.096595    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 09 20:53:38.102907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.101465    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"]
Sep 09 20:53:38.108119 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.107678    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"]
Sep 09 20:53:38.122799 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.121138    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hmvvb"]
Sep 09 20:53:38.125231 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.125184    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.127545 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.127523    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 20:53:38.128525 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.128201    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2sv5v\""
Sep 09 20:53:38.128986 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.128802    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 20:53:38.128986 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.128887    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 20:53:38.164389 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.164357    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:38.196556 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.196458    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.196556 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.196518    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.197008 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.196979    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.197177 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197156    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87c8906a-3840-4dcf-afa7-3e85753b28a0-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.197252 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197191    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl9zv\" (UniqueName: \"kubernetes.io/projected/87c8906a-3840-4dcf-afa7-3e85753b28a0-kube-api-access-kl9zv\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.197252 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197217    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.197252 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197241    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.197432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.197432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197315    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.197432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197342    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g2d4\" (UniqueName: \"kubernetes.io/projected/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-api-access-7g2d4\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.197432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.197637 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.197464    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.198915 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.198016    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.198915 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.198146    2571 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 09 20:53:38.198915 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.198218    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-tls podName:fe08f0b6-e8b7-4ede-85d3-489dfedde85c nodeName:}" failed. No retries permitted until 2026-09-09 20:53:38.698199184 +0000 UTC m=+312.668765880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-7kw6z" (UID: "fe08f0b6-e8b7-4ede-85d3-489dfedde85c") : secret "kube-state-metrics-tls" not found
Sep 09 20:53:38.198915 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.198726    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87c8906a-3840-4dcf-afa7-3e85753b28a0-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.198915 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.198814    2571 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 09 20:53:38.198915 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.198872    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls podName:87c8906a-3840-4dcf-afa7-3e85753b28a0 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:38.698855721 +0000 UTC m=+312.669422421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-r6dlw" (UID: "87c8906a-3840-4dcf-afa7-3e85753b28a0") : secret "openshift-state-metrics-tls" not found
Sep 09 20:53:38.201902 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.201877    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.202238 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.202220    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.209636 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.209586    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl9zv\" (UniqueName: \"kubernetes.io/projected/87c8906a-3840-4dcf-afa7-3e85753b28a0-kube-api-access-kl9zv\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.218157 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.218137    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g2d4\" (UniqueName: \"kubernetes.io/projected/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-api-access-7g2d4\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.297469 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.297426    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:38.297885 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.297858    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-sys\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.298003 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.297924    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-textfile\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.298003 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.297955    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-wtmp\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.298124 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.297992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-metrics-client-ca\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.298124 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.298068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-root\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.298185 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.298131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c48dd\" (UniqueName: \"kubernetes.io/projected/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-kube-api-access-c48dd\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.298246 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.298205    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.298306 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.298265    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.298306 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.298289    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.397183 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.397065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:38.398863 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.398832    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.399060 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.398877    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.399165 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.399072    2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 20:53:38.399223 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.399191    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls podName:e3fd2e5a-8d20-40c8-8d19-c958982e39c9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:38.899169518 +0000 UTC m=+312.869736219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls") pod "node-exporter-hmvvb" (UID: "e3fd2e5a-8d20-40c8-8d19-c958982e39c9") : secret "node-exporter-tls" not found
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399325    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-sys\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399435    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-textfile\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399468    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-wtmp\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399517    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-metrics-client-ca\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399534    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399605    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-root\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399650    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c48dd\" (UniqueName: \"kubernetes.io/projected/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-kube-api-access-c48dd\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399889    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-sys\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399903    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-textfile\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.399977    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-root\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400672 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.400041    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-wtmp\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.400672 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.400242    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-metrics-client-ca\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.403275 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.403229    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.409808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.409778    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c48dd\" (UniqueName: \"kubernetes.io/projected/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-kube-api-access-c48dd\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.505664 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.505478    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:38.702114 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.702029    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:38.702564 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.702197    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.702564 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.702212    2571 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 09 20:53:38.702564 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.702293    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls podName:87c8906a-3840-4dcf-afa7-3e85753b28a0 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:39.702268927 +0000 UTC m=+313.672835642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-r6dlw" (UID: "87c8906a-3840-4dcf-afa7-3e85753b28a0") : secret "openshift-state-metrics-tls" not found
Sep 09 20:53:38.705452 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.705424    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fe08f0b6-e8b7-4ede-85d3-489dfedde85c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-7kw6z\" (UID: \"fe08f0b6-e8b7-4ede-85d3-489dfedde85c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.725539 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.725503    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"
Sep 09 20:53:38.782810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.782728    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:38.903536 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:38.903489    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:38.903708 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.903656    2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 20:53:38.903780 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:38.903733    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls podName:e3fd2e5a-8d20-40c8-8d19-c958982e39c9 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:39.903710869 +0000 UTC m=+313.874277564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls") pod "node-exporter-hmvvb" (UID: "e3fd2e5a-8d20-40c8-8d19-c958982e39c9") : secret "node-exporter-tls" not found
Sep 09 20:53:39.167270 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.167161    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:39.270461 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.270428    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:39.286124 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.286090    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 20:53:39.290704 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.290670    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.293338 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.293311    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 09 20:53:39.293638 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.293576    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-xfgdf\""
Sep 09 20:53:39.293638 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.293586    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 09 20:53:39.293861 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.293836    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 09 20:53:39.293861 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.293843    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 09 20:53:39.294237 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.294219    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 09 20:53:39.294464 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.294446    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 09 20:53:39.294630 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.294615    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 09 20:53:39.294713 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.294640    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 09 20:53:39.294774 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.294659    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 09 20:53:39.305486 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.305466    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 20:53:39.307734 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.307708    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llf9m\" (UniqueName: \"kubernetes.io/projected/b491ab5c-08a4-4dd0-8073-176059571787-kube-api-access-llf9m\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.307844 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.307748    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.307844 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.307803    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b491ab5c-08a4-4dd0-8073-176059571787-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.307860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-config-volume\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.307931    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.307993    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.308054    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b491ab5c-08a4-4dd0-8073-176059571787-config-out\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.308110    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.308135    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-web-config\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.308157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.308178    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.308288    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.308613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.308311    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.353875 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.353824    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:39.409155 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409106    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llf9m\" (UniqueName: \"kubernetes.io/projected/b491ab5c-08a4-4dd0-8073-176059571787-kube-api-access-llf9m\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409155 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409166    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409230    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b491ab5c-08a4-4dd0-8073-176059571787-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409257    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-config-volume\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409293    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409336    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409375    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b491ab5c-08a4-4dd0-8073-176059571787-config-out\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409400    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409604 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409425    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-web-config\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409604 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409449    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409604 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409491    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409604 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409597    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.409747 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409624    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.410210 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:39.409790    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-trusted-ca-bundle podName:b491ab5c-08a4-4dd0-8073-176059571787 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:39.909767574 +0000 UTC m=+313.880334275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b491ab5c-08a4-4dd0-8073-176059571787") : configmap references non-existent config key: ca-bundle.crt
Sep 09 20:53:39.410210 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.409976    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.411000 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.410905    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.413432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.413369    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.413539 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.413477    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.414060 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.414016    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b491ab5c-08a4-4dd0-8073-176059571787-config-out\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.414635 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.414597    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b491ab5c-08a4-4dd0-8073-176059571787-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.414635 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.414616    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-config-volume\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.414778 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.414711    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-web-config\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.415257 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.415232    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.415654 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.415630    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.416647 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.416621    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b491ab5c-08a4-4dd0-8073-176059571787-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.424893 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.424840    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llf9m\" (UniqueName: \"kubernetes.io/projected/b491ab5c-08a4-4dd0-8073-176059571787-kube-api-access-llf9m\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.560191 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.560154    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:53:39.567835 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.567792    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:39.629398 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.629360    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:39.629588 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.629430    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:39.635479 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.635452    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:39.657473 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.657436    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:39.713023 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.712587    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:39.715504 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.715478    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/87c8906a-3840-4dcf-afa7-3e85753b28a0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-r6dlw\" (UID: \"87c8906a-3840-4dcf-afa7-3e85753b28a0\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:39.914086 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.914040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.914280 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.914111    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:39.915014 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.914989    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b491ab5c-08a4-4dd0-8073-176059571787-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b491ab5c-08a4-4dd0-8073-176059571787\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:39.916850 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.916822    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3fd2e5a-8d20-40c8-8d19-c958982e39c9-node-exporter-tls\") pod \"node-exporter-hmvvb\" (UID: \"e3fd2e5a-8d20-40c8-8d19-c958982e39c9\") " pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:39.919701 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.919681    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"
Sep 09 20:53:39.945210 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.945178    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hmvvb"
Sep 09 20:53:39.992848 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:39.992741    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:40.092916 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.092885    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:40.202055 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.202015    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 20:53:40.224111 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.224067    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-75498d46-bxr62"]
Sep 09 20:53:40.228207 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.228184    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.230771 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.230729    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 09 20:53:40.231220 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.231138    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-shjbz\""
Sep 09 20:53:40.231220 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.231193    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 09 20:53:40.231398 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.231200    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 09 20:53:40.231398 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.231294    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 09 20:53:40.232005 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.231984    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 09 20:53:40.232265 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.232246    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dcr8gspfcfvl3\""
Sep 09 20:53:40.244101 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.244019    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-75498d46-bxr62"]
Sep 09 20:53:40.318004 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.317960    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97c265a5-a54d-4ed9-810b-9d81eb0fec70-metrics-client-ca\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.318212 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.318038    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5p2k\" (UniqueName: \"kubernetes.io/projected/97c265a5-a54d-4ed9-810b-9d81eb0fec70-kube-api-access-r5p2k\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.318212 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.318099    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.318212 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.318140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-tls\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.318212 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.318176    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.318212 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.318208    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.318444 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.318254    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-grpc-tls\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.318444 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.318338    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.419550 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.419516    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97c265a5-a54d-4ed9-810b-9d81eb0fec70-metrics-client-ca\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.419740 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.419576    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5p2k\" (UniqueName: \"kubernetes.io/projected/97c265a5-a54d-4ed9-810b-9d81eb0fec70-kube-api-access-r5p2k\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.419740 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.419605    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.419740 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.419642    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-tls\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.419740 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.419679    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.419740 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.419704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.420007 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.419894    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-grpc-tls\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.420007 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.419961    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.420362 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.420336    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97c265a5-a54d-4ed9-810b-9d81eb0fec70-metrics-client-ca\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.423260 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.423199    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-tls\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.423260 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.423220    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.423607 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.423578    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.423694 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.423645    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.423694 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.423679    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.423923 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.423893    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/97c265a5-a54d-4ed9-810b-9d81eb0fec70-secret-grpc-tls\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.428956 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.428933    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5p2k\" (UniqueName: \"kubernetes.io/projected/97c265a5-a54d-4ed9-810b-9d81eb0fec70-kube-api-access-r5p2k\") pod \"thanos-querier-75498d46-bxr62\" (UID: \"97c265a5-a54d-4ed9-810b-9d81eb0fec70\") " pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.539748 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.539650    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:40.628579 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:40.628550    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:53:41.118219 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:41.118184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:41.219346 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:41.219300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:41.320413 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:41.320327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:42.418860 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.418800    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"]
Sep 09 20:53:42.423658 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.423638    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.426273 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.426251    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-20je0oekppsk3\""
Sep 09 20:53:42.426858 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.426836    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 09 20:53:42.426858 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.426850    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 09 20:53:42.427152 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.427134    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 20:53:42.427240 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.427222    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sp9bc\""
Sep 09 20:53:42.427517 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.427499    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 09 20:53:42.434765 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.434744    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"]
Sep 09 20:53:42.440363 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.440341    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-secret-metrics-server-tls\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.440459 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.440375    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5fbc453e-193a-41d6-b01d-87365ca9b6a1-metrics-server-audit-profiles\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.440459 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.440429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5fbc453e-193a-41d6-b01d-87365ca9b6a1-audit-log\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.440574 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.440464    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4ftl\" (UniqueName: \"kubernetes.io/projected/5fbc453e-193a-41d6-b01d-87365ca9b6a1-kube-api-access-l4ftl\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.440574 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.440491    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fbc453e-193a-41d6-b01d-87365ca9b6a1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.440680 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.440573    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-secret-metrics-server-client-certs\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.440680 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.440662    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-client-ca-bundle\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.530066 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.530010    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:42.541270 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.541239    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-client-ca-bundle\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.541432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.541288    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-secret-metrics-server-tls\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.541432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.541307    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5fbc453e-193a-41d6-b01d-87365ca9b6a1-metrics-server-audit-profiles\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.541432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.541350    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5fbc453e-193a-41d6-b01d-87365ca9b6a1-audit-log\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.541432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.541377    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4ftl\" (UniqueName: \"kubernetes.io/projected/5fbc453e-193a-41d6-b01d-87365ca9b6a1-kube-api-access-l4ftl\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.541432 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.541414    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fbc453e-193a-41d6-b01d-87365ca9b6a1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.541743 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.541447    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-secret-metrics-server-client-certs\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.542253 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.542172    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5fbc453e-193a-41d6-b01d-87365ca9b6a1-audit-log\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.542457 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.542432    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fbc453e-193a-41d6-b01d-87365ca9b6a1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.542773 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.542751    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5fbc453e-193a-41d6-b01d-87365ca9b6a1-metrics-server-audit-profiles\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.544399 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.544372    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-secret-metrics-server-client-certs\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.544693 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.544662    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-secret-metrics-server-tls\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.544862 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.544838    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5fbc453e-193a-41d6-b01d-87365ca9b6a1-client-ca-bundle\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.550281 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.550258    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4ftl\" (UniqueName: \"kubernetes.io/projected/5fbc453e-193a-41d6-b01d-87365ca9b6a1-kube-api-access-l4ftl\") pod \"metrics-server-78b8c7bc6b-lf6vw\" (UID: \"5fbc453e-193a-41d6-b01d-87365ca9b6a1\") " pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.629895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.629844    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:42.735767 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.735724    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:53:42.872023 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.871976    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"]
Sep 09 20:53:42.875607 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.875583    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:42.878037 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.877994    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 20:53:42.878174 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.878126    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vww42\""
Sep 09 20:53:42.883628 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.883605    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:42.887029 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.887004    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"]
Sep 09 20:53:42.944611 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.944579    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77544acd-b941-4893-9ead-4b53ca7fcc99-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-2n8jg\" (UID: \"77544acd-b941-4893-9ead-4b53ca7fcc99\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:42.944611 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.944622    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6drqp\" (UniqueName: \"kubernetes.io/projected/77544acd-b941-4893-9ead-4b53ca7fcc99-kube-api-access-6drqp\") pod \"monitoring-plugin-85bbbbbf4-2n8jg\" (UID: \"77544acd-b941-4893-9ead-4b53ca7fcc99\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:42.982558 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:42.982513    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:43.045659 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.045562    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77544acd-b941-4893-9ead-4b53ca7fcc99-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-2n8jg\" (UID: \"77544acd-b941-4893-9ead-4b53ca7fcc99\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:43.045659 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.045616    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6drqp\" (UniqueName: \"kubernetes.io/projected/77544acd-b941-4893-9ead-4b53ca7fcc99-kube-api-access-6drqp\") pod \"monitoring-plugin-85bbbbbf4-2n8jg\" (UID: \"77544acd-b941-4893-9ead-4b53ca7fcc99\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:43.045884 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:43.045742    2571 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 09 20:53:43.045884 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:53:43.045820    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77544acd-b941-4893-9ead-4b53ca7fcc99-monitoring-plugin-cert podName:77544acd-b941-4893-9ead-4b53ca7fcc99 nodeName:}" failed. No retries permitted until 2026-09-09 20:53:43.54579852 +0000 UTC m=+317.516365228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/77544acd-b941-4893-9ead-4b53ca7fcc99-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-2n8jg" (UID: "77544acd-b941-4893-9ead-4b53ca7fcc99") : secret "monitoring-plugin-cert" not found
Sep 09 20:53:43.057763 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.057733    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6drqp\" (UniqueName: \"kubernetes.io/projected/77544acd-b941-4893-9ead-4b53ca7fcc99-kube-api-access-6drqp\") pod \"monitoring-plugin-85bbbbbf4-2n8jg\" (UID: \"77544acd-b941-4893-9ead-4b53ca7fcc99\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:43.113639 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:43.113599    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3fd2e5a_8d20_40c8_8d19_c958982e39c9.slice/crio-0b5900feca859a24738762b7b11af7284b8a4bcd532ad775a61c8cd6a3e65495 WatchSource:0}: Error finding container 0b5900feca859a24738762b7b11af7284b8a4bcd532ad775a61c8cd6a3e65495: Status 404 returned error can't find the container with id 0b5900feca859a24738762b7b11af7284b8a4bcd532ad775a61c8cd6a3e65495
Sep 09 20:53:43.243128 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.243054    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:43.394498 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.394405    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-75498d46-bxr62"]
Sep 09 20:53:43.396495 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:43.396465    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97c265a5_a54d_4ed9_810b_9d81eb0fec70.slice/crio-306fe29d0a83624f61b76f02a2c3ffd31bdf49bd49a2b48ef3efff0362665cff WatchSource:0}: Error finding container 306fe29d0a83624f61b76f02a2c3ffd31bdf49bd49a2b48ef3efff0362665cff: Status 404 returned error can't find the container with id 306fe29d0a83624f61b76f02a2c3ffd31bdf49bd49a2b48ef3efff0362665cff
Sep 09 20:53:43.439254 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.439217    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z"]
Sep 09 20:53:43.442753 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:43.442722    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe08f0b6_e8b7_4ede_85d3_489dfedde85c.slice/crio-75fdf7bdcadd9c7447c4431596c97ae16a12a417f731c57bdb4c055e33a5ff1d WatchSource:0}: Error finding container 75fdf7bdcadd9c7447c4431596c97ae16a12a417f731c57bdb4c055e33a5ff1d: Status 404 returned error can't find the container with id 75fdf7bdcadd9c7447c4431596c97ae16a12a417f731c57bdb4c055e33a5ff1d
Sep 09 20:53:43.447918 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.447882    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54bf487c68-rj7j6"]
Sep 09 20:53:43.451808 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.451774    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw"]
Sep 09 20:53:43.452009 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:43.451972    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58dcb4e_1478_4c22_9ba1_10bd526d6946.slice/crio-7d6bb2577db8b6667f0fcd44caf7bcb5cb1cf7ca64650c5b7d4514de89e8bc9f WatchSource:0}: Error finding container 7d6bb2577db8b6667f0fcd44caf7bcb5cb1cf7ca64650c5b7d4514de89e8bc9f: Status 404 returned error can't find the container with id 7d6bb2577db8b6667f0fcd44caf7bcb5cb1cf7ca64650c5b7d4514de89e8bc9f
Sep 09 20:53:43.455660 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:43.455638    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c8906a_3840_4dcf_afa7_3e85753b28a0.slice/crio-e4dc39b968d10d25c46420c64f9047c37603712ca49568e071961a5296c2762a WatchSource:0}: Error finding container e4dc39b968d10d25c46420c64f9047c37603712ca49568e071961a5296c2762a: Status 404 returned error can't find the container with id e4dc39b968d10d25c46420c64f9047c37603712ca49568e071961a5296c2762a
Sep 09 20:53:43.471499 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.471434    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54bf487c68-rj7j6"]
Sep 09 20:53:43.504494 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.504417    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"]
Sep 09 20:53:43.507873 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:43.507840    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fbc453e_193a_41d6_b01d_87365ca9b6a1.slice/crio-7cfe8d7ee8bb3cf184637d6afa41965c91d495d5254904e85ca28c4fdcc5337a WatchSource:0}: Error finding container 7cfe8d7ee8bb3cf184637d6afa41965c91d495d5254904e85ca28c4fdcc5337a: Status 404 returned error can't find the container with id 7cfe8d7ee8bb3cf184637d6afa41965c91d495d5254904e85ca28c4fdcc5337a
Sep 09 20:53:43.514691 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.514656    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 20:53:43.518035 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:43.518005    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb491ab5c_08a4_4dd0_8073_176059571787.slice/crio-486dc209015656f2fe020babcdb257f438c39e917236c46f064068f24b1fe417 WatchSource:0}: Error finding container 486dc209015656f2fe020babcdb257f438c39e917236c46f064068f24b1fe417: Status 404 returned error can't find the container with id 486dc209015656f2fe020babcdb257f438c39e917236c46f064068f24b1fe417
Sep 09 20:53:43.553392 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.553363    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77544acd-b941-4893-9ead-4b53ca7fcc99-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-2n8jg\" (UID: \"77544acd-b941-4893-9ead-4b53ca7fcc99\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:43.556428 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.556401    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/77544acd-b941-4893-9ead-4b53ca7fcc99-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-2n8jg\" (UID: \"77544acd-b941-4893-9ead-4b53ca7fcc99\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:43.646094 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.646020    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw" event={"ID":"5fbc453e-193a-41d6-b01d-87365ca9b6a1","Type":"ContainerStarted","Data":"7cfe8d7ee8bb3cf184637d6afa41965c91d495d5254904e85ca28c4fdcc5337a"}
Sep 09 20:53:43.647508 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.647433    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b491ab5c-08a4-4dd0-8073-176059571787","Type":"ContainerStarted","Data":"486dc209015656f2fe020babcdb257f438c39e917236c46f064068f24b1fe417"}
Sep 09 20:53:43.649556 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.649500    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmvvb" event={"ID":"e3fd2e5a-8d20-40c8-8d19-c958982e39c9","Type":"ContainerStarted","Data":"0b5900feca859a24738762b7b11af7284b8a4bcd532ad775a61c8cd6a3e65495"}
Sep 09 20:53:43.651487 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.651462    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bf487c68-rj7j6" event={"ID":"d58dcb4e-1478-4c22-9ba1-10bd526d6946","Type":"ContainerStarted","Data":"0c840d798e51df21db475896b5b1218a26c1381cb2b6a6bcf00dac2b0880c94e"}
Sep 09 20:53:43.651583 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.651504    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bf487c68-rj7j6" event={"ID":"d58dcb4e-1478-4c22-9ba1-10bd526d6946","Type":"ContainerStarted","Data":"7d6bb2577db8b6667f0fcd44caf7bcb5cb1cf7ca64650c5b7d4514de89e8bc9f"}
Sep 09 20:53:43.655694 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.655672    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z" event={"ID":"fe08f0b6-e8b7-4ede-85d3-489dfedde85c","Type":"ContainerStarted","Data":"75fdf7bdcadd9c7447c4431596c97ae16a12a417f731c57bdb4c055e33a5ff1d"}
Sep 09 20:53:43.657860 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.657829    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-5n6ft" event={"ID":"270c8650-2237-4f09-ac0e-9d7c6b803e17","Type":"ContainerStarted","Data":"e9dbd2b56eee915ebc01768bbb54d1edc005f2bec47790d22764901dc81b0067"}
Sep 09 20:53:43.658827 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.658807    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-5n6ft"
Sep 09 20:53:43.662484 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.662436    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw" event={"ID":"87c8906a-3840-4dcf-afa7-3e85753b28a0","Type":"ContainerStarted","Data":"a38b0b99916f47844c460c8186de9fe53e98572db1bceb419c8db341d0e3c915"}
Sep 09 20:53:43.662668 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.662619    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw" event={"ID":"87c8906a-3840-4dcf-afa7-3e85753b28a0","Type":"ContainerStarted","Data":"92a88fe10e447bf5152d11592c883fcc3adbb3037bfa0bac01a0d6ceeac980c9"}
Sep 09 20:53:43.662864 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.662813    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw" event={"ID":"87c8906a-3840-4dcf-afa7-3e85753b28a0","Type":"ContainerStarted","Data":"e4dc39b968d10d25c46420c64f9047c37603712ca49568e071961a5296c2762a"}
Sep 09 20:53:43.663834 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.663739    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-5n6ft"
Sep 09 20:53:43.663834 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.663772    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75498d46-bxr62" event={"ID":"97c265a5-a54d-4ed9-810b-9d81eb0fec70","Type":"ContainerStarted","Data":"306fe29d0a83624f61b76f02a2c3ffd31bdf49bd49a2b48ef3efff0362665cff"}
Sep 09 20:53:43.679471 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.677778    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54bf487c68-rj7j6" podStartSLOduration=6.677759319 podStartE2EDuration="6.677759319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:43.677252166 +0000 UTC m=+317.647818885" watchObservedRunningTime="2026-09-09 20:53:43.677759319 +0000 UTC m=+317.648326040"
Sep 09 20:53:43.698570 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.697304    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-5n6ft" podStartSLOduration=1.257571422 podStartE2EDuration="18.697281169s" totalImagesPullingTime="17.439709747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:43.695780673 +0000 UTC m=+317.666347393" watchObservedRunningTime="2026-09-09 20:53:43.697281169 +0000 UTC m=+317.667847887"
Sep 09 20:53:43.787916 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:43.787668    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:44.084917 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:44.084617    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"]
Sep 09 20:53:44.464909 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:44.464792    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:44.671903 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:53:44.671500    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77544acd_b941_4893_9ead_4b53ca7fcc99.slice/crio-c0a5759dc180b6f2318eaffdc11bc285b61df8976c06961540b20c652d95f8b9 WatchSource:0}: Error finding container c0a5759dc180b6f2318eaffdc11bc285b61df8976c06961540b20c652d95f8b9: Status 404 returned error can't find the container with id c0a5759dc180b6f2318eaffdc11bc285b61df8976c06961540b20c652d95f8b9
Sep 09 20:53:45.259060 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:45.258987    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:45.361141 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:45.361028    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:45.642034 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:45.641978    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:45.683130 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:45.683046    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg" event={"ID":"77544acd-b941-4893-9ead-4b53ca7fcc99","Type":"ContainerStarted","Data":"c0a5759dc180b6f2318eaffdc11bc285b61df8976c06961540b20c652d95f8b9"}
Sep 09 20:53:45.737880 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:45.737823    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:46.280554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:46.280451    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:46.386201 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:46.386035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:47.969981 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:47.969944    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:53:47.979853 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:47.979818    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bb5b4df67-6qsn4"]
Sep 09 20:53:48.110774 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:48.110632    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:48.209937 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:48.209898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:49.706862 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.705764    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw" event={"ID":"5fbc453e-193a-41d6-b01d-87365ca9b6a1","Type":"ContainerStarted","Data":"23bd1ab8c18896c255b31c376a21214709771f041358dc491dfde7d43089472c"}
Sep 09 20:53:49.710761 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.710697    2571 generic.go:356] "Generic (PLEG): container finished" podID="b491ab5c-08a4-4dd0-8073-176059571787" containerID="90b90bef0a2a883982aa597d1eec754152274c1dd551e43383f8058a6a6631a8" exitCode=0
Sep 09 20:53:49.710878 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.710807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b491ab5c-08a4-4dd0-8073-176059571787","Type":"ContainerDied","Data":"90b90bef0a2a883982aa597d1eec754152274c1dd551e43383f8058a6a6631a8"}
Sep 09 20:53:49.714768 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.714421    2571 generic.go:356] "Generic (PLEG): container finished" podID="e3fd2e5a-8d20-40c8-8d19-c958982e39c9" containerID="f94d0225101c059db280826c7c6c7bbfbf0bb120e3838d854284b42b95dbc796" exitCode=0
Sep 09 20:53:49.714768 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.714513    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmvvb" event={"ID":"e3fd2e5a-8d20-40c8-8d19-c958982e39c9","Type":"ContainerDied","Data":"f94d0225101c059db280826c7c6c7bbfbf0bb120e3838d854284b42b95dbc796"}
Sep 09 20:53:49.724211 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.723901    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z" event={"ID":"fe08f0b6-e8b7-4ede-85d3-489dfedde85c","Type":"ContainerStarted","Data":"3d41010503bf18251ea90760e337475011a8b2495a790834b68109c5825a4987"}
Sep 09 20:53:49.731479 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.731216    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw" podStartSLOduration=2.083231476 podStartE2EDuration="7.731198516s" totalImagesPullingTime="5.64796704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:49.728610118 +0000 UTC m=+323.699176836" watchObservedRunningTime="2026-09-09 20:53:49.731198516 +0000 UTC m=+323.701765231"
Sep 09 20:53:49.732295 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.732268    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw" event={"ID":"87c8906a-3840-4dcf-afa7-3e85753b28a0","Type":"ContainerStarted","Data":"120e43b2c1a1c75c4b379ab56cc2f6efaba72ce696f2c93252133b73b46df8c6"}
Sep 09 20:53:49.734797 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.734769    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75498d46-bxr62" event={"ID":"97c265a5-a54d-4ed9-810b-9d81eb0fec70","Type":"ContainerStarted","Data":"40f37b20e84d9ff7fa231ba4bc994fce07408bbf86e5a0d570250022bb857aed"}
Sep 09 20:53:49.734895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.734844    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75498d46-bxr62" event={"ID":"97c265a5-a54d-4ed9-810b-9d81eb0fec70","Type":"ContainerStarted","Data":"ae8c29b0443f055d777514d9979f010064ae2a41b08e981af9ac7dcb025abc20"}
Sep 09 20:53:49.814792 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:49.813426    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-r6dlw" podStartSLOduration=6.3024507849999996 podStartE2EDuration="11.813403189s" totalImagesPullingTime="5.510952404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:49.810488579 +0000 UTC m=+323.781055299" watchObservedRunningTime="2026-09-09 20:53:49.813403189 +0000 UTC m=+323.783969908"
Sep 09 20:53:50.746750 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.745486    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg" event={"ID":"77544acd-b941-4893-9ead-4b53ca7fcc99","Type":"ContainerStarted","Data":"67b79e2369c7ffdad6347519c91d9758009719198a2897855824fa10558e236b"}
Sep 09 20:53:50.746750 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.746693    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:50.753806 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.753582    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z" event={"ID":"fe08f0b6-e8b7-4ede-85d3-489dfedde85c","Type":"ContainerStarted","Data":"277217c6518a5c270155dc799326047660fdb9fad7c4fc97e9084eb45129f3fa"}
Sep 09 20:53:50.753806 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.753641    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z" event={"ID":"fe08f0b6-e8b7-4ede-85d3-489dfedde85c","Type":"ContainerStarted","Data":"6403dfff4c33b0223d0da8f654a616963725cac20ca22d069b83f43023720ec4"}
Sep 09 20:53:50.754920 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.754863    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg"
Sep 09 20:53:50.760774 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.760742    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75498d46-bxr62" event={"ID":"97c265a5-a54d-4ed9-810b-9d81eb0fec70","Type":"ContainerStarted","Data":"7dfd34a8d1df882e75aead3f79cec2fed01e5d5b5ae0a61fe461ffa141559b69"}
Sep 09 20:53:50.767863 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.766937    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmvvb" event={"ID":"e3fd2e5a-8d20-40c8-8d19-c958982e39c9","Type":"ContainerStarted","Data":"de049462290318c8d69e7a23940e5a1b37ea2afa11447a345036594ea20c05bb"}
Sep 09 20:53:50.767863 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.766993    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmvvb" event={"ID":"e3fd2e5a-8d20-40c8-8d19-c958982e39c9","Type":"ContainerStarted","Data":"099e2ce4b273127197d337bc3bf92a13d32ea42714cff9cf495754f1ee2b61ed"}
Sep 09 20:53:50.777421 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.776948    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-2n8jg" podStartSLOduration=3.877746867 podStartE2EDuration="8.776931935s" totalImagesPullingTime="4.899185068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:50.776910817 +0000 UTC m=+324.747477537" watchObservedRunningTime="2026-09-09 20:53:50.776931935 +0000 UTC m=+324.747498653"
Sep 09 20:53:50.803818 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.803758    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-7kw6z" podStartSLOduration=7.094915153 podStartE2EDuration="12.803738244s" totalImagesPullingTime="5.708823091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:50.802499048 +0000 UTC m=+324.773065767" watchObservedRunningTime="2026-09-09 20:53:50.803738244 +0000 UTC m=+324.774304961"
Sep 09 20:53:50.849039 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:50.846628    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hmvvb" podStartSLOduration=7.44494146 podStartE2EDuration="12.846599254s" totalImagesPullingTime="5.287427138s" totalInitContainerRuntime="114.230656ms" isStatefulPod=true podCreationTimestamp="2026-09-09 20:53:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:50.846115557 +0000 UTC m=+324.816682275" watchObservedRunningTime="2026-09-09 20:53:50.846599254 +0000 UTC m=+324.817165972"
Sep 09 20:53:52.701515 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.701470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:52.779707 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.779665    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75498d46-bxr62" event={"ID":"97c265a5-a54d-4ed9-810b-9d81eb0fec70","Type":"ContainerStarted","Data":"c7ffe63f6565eca1e2726e41bfcec0c6485495a8ff393114c54fa88898c04723"}
Sep 09 20:53:52.779899 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.779797    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:53:52.779899 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.779844    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75498d46-bxr62" event={"ID":"97c265a5-a54d-4ed9-810b-9d81eb0fec70","Type":"ContainerStarted","Data":"78293a435590da20855080a735638db64fa25d1007c6b786b28ad8d6f4c0065f"}
Sep 09 20:53:52.779899 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.779884    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75498d46-bxr62" event={"ID":"97c265a5-a54d-4ed9-810b-9d81eb0fec70","Type":"ContainerStarted","Data":"f07ec47956527c53e056dec0494e9fa510464f5f7ca0f2f053449903ffaf41e2"}
Sep 09 20:53:52.782632 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.782598    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b491ab5c-08a4-4dd0-8073-176059571787","Type":"ContainerStarted","Data":"44112420cb9e3e211597bba1a04918e1923682bc9336d84d46571f0120c69fe6"}
Sep 09 20:53:52.782760 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.782655    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b491ab5c-08a4-4dd0-8073-176059571787","Type":"ContainerStarted","Data":"4f293820bc4aa0b31e037888d549d2d18a435d7ee6b50119de097a302c0446ff"}
Sep 09 20:53:52.782760 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.782680    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b491ab5c-08a4-4dd0-8073-176059571787","Type":"ContainerStarted","Data":"d40463721cf2e9fe22e6784a591acbddd3581ea2ffe669f130cea096f59deb7d"}
Sep 09 20:53:52.782760 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.782710    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b491ab5c-08a4-4dd0-8073-176059571787","Type":"ContainerStarted","Data":"7c38c5f328c0e10def3c2fb1783a1a06dc50fa1549d6a36faef68660b57f9e2d"}
Sep 09 20:53:52.782760 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.782734    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b491ab5c-08a4-4dd0-8073-176059571787","Type":"ContainerStarted","Data":"31b6859bf8d347c718dcbe445775a6fcb033c04206fdb09fc3558ce206cb7dce"}
Sep 09 20:53:52.803636 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.803603    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:52.810530 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:52.810473    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-75498d46-bxr62" podStartSLOduration=4.829912074 podStartE2EDuration="12.810457146s" totalImagesPullingTime="7.980545072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:52.807272955 +0000 UTC m=+326.777839672" watchObservedRunningTime="2026-09-09 20:53:52.810457146 +0000 UTC m=+326.781023892"
Sep 09 20:53:53.789972 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:53.789938    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b491ab5c-08a4-4dd0-8073-176059571787","Type":"ContainerStarted","Data":"fe49ecc57d5d21657a041a3874bd927f6de2ef671dff03afd79f4b67668fab34"}
Sep 09 20:53:53.823891 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:53.823700    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.790240861 podStartE2EDuration="14.823680149s" totalImagesPullingTime="7.999318214s" totalInitContainerRuntime="34.121074ms" isStatefulPod=false podCreationTimestamp="2026-09-09 20:53:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:53:53.822644679 +0000 UTC m=+327.793211394" watchObservedRunningTime="2026-09-09 20:53:53.823680149 +0000 UTC m=+327.794246867"
Sep 09 20:53:53.959309 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:53.959279    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5fb94fb5-xx9nj"]
Sep 09 20:53:54.091044 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:54.090951    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:54.298400 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:54.298366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:54.433368 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:54.433270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:53:58.796776 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:53:58.796749    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-75498d46-bxr62"
Sep 09 20:54:00.735760 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:00.735715    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:00.832700 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:00.832648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:02.736416 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:02.736383    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:54:02.736792 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:02.736442    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:54:08.680820 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:08.680776    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54bf487c68-rj7j6" podUID="d58dcb4e-1478-4c22-9ba1-10bd526d6946" containerName="console" containerID="cri-o://0c840d798e51df21db475896b5b1218a26c1381cb2b6a6bcf00dac2b0880c94e" gracePeriod=15
Sep 09 20:54:08.865236 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:08.865062    2571 generic.go:356] "Generic (PLEG): container finished" podID="d58dcb4e-1478-4c22-9ba1-10bd526d6946" containerID="0c840d798e51df21db475896b5b1218a26c1381cb2b6a6bcf00dac2b0880c94e" exitCode=0
Sep 09 20:54:08.865236 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:08.865120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bf487c68-rj7j6" event={"ID":"d58dcb4e-1478-4c22-9ba1-10bd526d6946","Type":"ContainerDied","Data":"0c840d798e51df21db475896b5b1218a26c1381cb2b6a6bcf00dac2b0880c94e"}
Sep 09 20:54:08.963514 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:08.963486    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:54:09.110312 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110273    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-trusted-ca-bundle\") pod \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") "
Sep 09 20:54:09.110312 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110314    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-config\") pod \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") "
Sep 09 20:54:09.110554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110340    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-oauth-serving-cert\") pod \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") "
Sep 09 20:54:09.110554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110367    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-service-ca\") pod \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") "
Sep 09 20:54:09.110554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110419    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-serving-cert\") pod \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") "
Sep 09 20:54:09.110554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110465    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cqnr\" (UniqueName: \"kubernetes.io/projected/d58dcb4e-1478-4c22-9ba1-10bd526d6946-kube-api-access-4cqnr\") pod \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") "
Sep 09 20:54:09.110554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110527    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-oauth-config\") pod \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\" (UID: \"d58dcb4e-1478-4c22-9ba1-10bd526d6946\") "
Sep 09 20:54:09.110825 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110794    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d58dcb4e-1478-4c22-9ba1-10bd526d6946" (UID: "d58dcb4e-1478-4c22-9ba1-10bd526d6946"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:09.110891 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110771    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d58dcb4e-1478-4c22-9ba1-10bd526d6946" (UID: "d58dcb4e-1478-4c22-9ba1-10bd526d6946"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:09.110891 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110882    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-config" (OuterVolumeSpecName: "console-config") pod "d58dcb4e-1478-4c22-9ba1-10bd526d6946" (UID: "d58dcb4e-1478-4c22-9ba1-10bd526d6946"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:09.110999 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.110981    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-service-ca" (OuterVolumeSpecName: "service-ca") pod "d58dcb4e-1478-4c22-9ba1-10bd526d6946" (UID: "d58dcb4e-1478-4c22-9ba1-10bd526d6946"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:09.112948 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.112916    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58dcb4e-1478-4c22-9ba1-10bd526d6946-kube-api-access-4cqnr" (OuterVolumeSpecName: "kube-api-access-4cqnr") pod "d58dcb4e-1478-4c22-9ba1-10bd526d6946" (UID: "d58dcb4e-1478-4c22-9ba1-10bd526d6946"). InnerVolumeSpecName "kube-api-access-4cqnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:09.112948 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.112921    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d58dcb4e-1478-4c22-9ba1-10bd526d6946" (UID: "d58dcb4e-1478-4c22-9ba1-10bd526d6946"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:09.113132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.112946    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d58dcb4e-1478-4c22-9ba1-10bd526d6946" (UID: "d58dcb4e-1478-4c22-9ba1-10bd526d6946"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:09.212246 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.212215    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-oauth-serving-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:09.212246 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.212246    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:09.212433 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.212255    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-serving-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:09.212433 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.212264    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cqnr\" (UniqueName: \"kubernetes.io/projected/d58dcb4e-1478-4c22-9ba1-10bd526d6946-kube-api-access-4cqnr\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:09.212433 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.212274    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-oauth-config\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:09.212433 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.212282    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-trusted-ca-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:09.212433 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.212290    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d58dcb4e-1478-4c22-9ba1-10bd526d6946-console-config\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:09.870984 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.870948    2571 generic.go:356] "Generic (PLEG): container finished" podID="caaa9f3d-3587-4330-ac7f-e3982c2c09a4" containerID="af38401729e2b81f6ad82c772ed3e37a0a1d38cff51f679396f0f4eed2e432cd" exitCode=0
Sep 09 20:54:09.871412 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.871030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg" event={"ID":"caaa9f3d-3587-4330-ac7f-e3982c2c09a4","Type":"ContainerDied","Data":"af38401729e2b81f6ad82c772ed3e37a0a1d38cff51f679396f0f4eed2e432cd"}
Sep 09 20:54:09.871462 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.871420    2571 scope.go:117] "RemoveContainer" containerID="af38401729e2b81f6ad82c772ed3e37a0a1d38cff51f679396f0f4eed2e432cd"
Sep 09 20:54:09.872523 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.872502    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54bf487c68-rj7j6" event={"ID":"d58dcb4e-1478-4c22-9ba1-10bd526d6946","Type":"ContainerDied","Data":"7d6bb2577db8b6667f0fcd44caf7bcb5cb1cf7ca64650c5b7d4514de89e8bc9f"}
Sep 09 20:54:09.872627 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.872560    2571 scope.go:117] "RemoveContainer" containerID="0c840d798e51df21db475896b5b1218a26c1381cb2b6a6bcf00dac2b0880c94e"
Sep 09 20:54:09.872627 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.872508    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54bf487c68-rj7j6"
Sep 09 20:54:09.908895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.908869    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54bf487c68-rj7j6"]
Sep 09 20:54:09.913931 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:09.913907    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54bf487c68-rj7j6"]
Sep 09 20:54:10.565028 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:10.564995    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d58dcb4e-1478-4c22-9ba1-10bd526d6946" path="/var/lib/kubelet/pods/d58dcb4e-1478-4c22-9ba1-10bd526d6946/volumes"
Sep 09 20:54:10.878769 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:10.878682    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-jlprg" event={"ID":"caaa9f3d-3587-4330-ac7f-e3982c2c09a4","Type":"ContainerStarted","Data":"df9e857f53e91bf1712fe09bde850892b66f439f5664221befd0f9a808c4e0e7"}
Sep 09 20:54:13.009134 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.009070    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" podUID="9b6dd829-83b3-4d95-9ee8-241bcdc87792" containerName="registry" containerID="cri-o://a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01" gracePeriod=30
Sep 09 20:54:13.253373 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.253343    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:54:13.353490 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353387    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5dpn\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-kube-api-access-m5dpn\") pod \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") "
Sep 09 20:54:13.353490 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353453    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-certificates\") pod \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") "
Sep 09 20:54:13.353714 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353501    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") pod \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") "
Sep 09 20:54:13.353714 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353554    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-installation-pull-secrets\") pod \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") "
Sep 09 20:54:13.353714 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353580    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b6dd829-83b3-4d95-9ee8-241bcdc87792-ca-trust-extracted\") pod \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") "
Sep 09 20:54:13.353714 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353611    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-bound-sa-token\") pod \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") "
Sep 09 20:54:13.353714 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353632    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-trusted-ca\") pod \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") "
Sep 09 20:54:13.353714 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353672    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-image-registry-private-configuration\") pod \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\" (UID: \"9b6dd829-83b3-4d95-9ee8-241bcdc87792\") "
Sep 09 20:54:13.353996 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.353907    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9b6dd829-83b3-4d95-9ee8-241bcdc87792" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:13.354650 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.354616    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9b6dd829-83b3-4d95-9ee8-241bcdc87792" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:13.356329 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.356296    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9b6dd829-83b3-4d95-9ee8-241bcdc87792" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:13.356435 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.356349    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-kube-api-access-m5dpn" (OuterVolumeSpecName: "kube-api-access-m5dpn") pod "9b6dd829-83b3-4d95-9ee8-241bcdc87792" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792"). InnerVolumeSpecName "kube-api-access-m5dpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:13.356435 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.356365    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9b6dd829-83b3-4d95-9ee8-241bcdc87792" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:13.356685 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.356667    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9b6dd829-83b3-4d95-9ee8-241bcdc87792" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:13.356740 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.356693    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9b6dd829-83b3-4d95-9ee8-241bcdc87792" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:13.362830 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.362804    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b6dd829-83b3-4d95-9ee8-241bcdc87792-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9b6dd829-83b3-4d95-9ee8-241bcdc87792" (UID: "9b6dd829-83b3-4d95-9ee8-241bcdc87792"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:54:13.454817 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.454763    2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-image-registry-private-configuration\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.454817 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.454808    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5dpn\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-kube-api-access-m5dpn\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.454817 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.454819    2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-certificates\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.454817 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.454830    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-registry-tls\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.454817 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.454839    2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9b6dd829-83b3-4d95-9ee8-241bcdc87792-installation-pull-secrets\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.455155 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.454848    2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9b6dd829-83b3-4d95-9ee8-241bcdc87792-ca-trust-extracted\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.455155 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.454856    2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9b6dd829-83b3-4d95-9ee8-241bcdc87792-bound-sa-token\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.455155 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.454864    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9b6dd829-83b3-4d95-9ee8-241bcdc87792-trusted-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:13.893690 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.893657    2571 generic.go:356] "Generic (PLEG): container finished" podID="9b6dd829-83b3-4d95-9ee8-241bcdc87792" containerID="a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01" exitCode=0
Sep 09 20:54:13.893869 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.893716    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4"
Sep 09 20:54:13.893869 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.893731    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" event={"ID":"9b6dd829-83b3-4d95-9ee8-241bcdc87792","Type":"ContainerDied","Data":"a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01"}
Sep 09 20:54:13.893869 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.893770    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bb5b4df67-6qsn4" event={"ID":"9b6dd829-83b3-4d95-9ee8-241bcdc87792","Type":"ContainerDied","Data":"28ebd1cfcdc31c5ec8c7966d6da9a264f3735d87590dcb9a342ecf94c5be8ae8"}
Sep 09 20:54:13.893869 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.893790    2571 scope.go:117] "RemoveContainer" containerID="a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01"
Sep 09 20:54:13.903195 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.903176    2571 scope.go:117] "RemoveContainer" containerID="a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01"
Sep 09 20:54:13.903483 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:54:13.903461    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01\": container with ID starting with a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01 not found: ID does not exist" containerID="a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01"
Sep 09 20:54:13.903555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.903492    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01"} err="failed to get container status \"a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01\": rpc error: code = NotFound desc = could not find container \"a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01\": container with ID starting with a15a8758d67de0436faa8027836b2cb5ebb30954a4e3658615923518e4d3de01 not found: ID does not exist"
Sep 09 20:54:13.915753 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.915732    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bb5b4df67-6qsn4"]
Sep 09 20:54:13.918793 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:13.918771    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7bb5b4df67-6qsn4"]
Sep 09 20:54:14.565506 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:14.565471    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6dd829-83b3-4d95-9ee8-241bcdc87792" path="/var/lib/kubelet/pods/9b6dd829-83b3-4d95-9ee8-241bcdc87792/volumes"
Sep 09 20:54:15.694471 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:15.694435    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:15.780781 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:15.780744    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:18.980737 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:18.980631    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c5fb94fb5-xx9nj" podUID="c40a6696-c9a7-4897-a094-7f30c33e6494" containerName="console" containerID="cri-o://8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3" gracePeriod=15
Sep 09 20:54:19.272314 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.272288    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:54:19.301728 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.301696    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzqpp\" (UniqueName: \"kubernetes.io/projected/c40a6696-c9a7-4897-a094-7f30c33e6494-kube-api-access-pzqpp\") pod \"c40a6696-c9a7-4897-a094-7f30c33e6494\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") "
Sep 09 20:54:19.301900 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.301752    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-serving-cert\") pod \"c40a6696-c9a7-4897-a094-7f30c33e6494\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") "
Sep 09 20:54:19.301900 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.301794    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-console-config\") pod \"c40a6696-c9a7-4897-a094-7f30c33e6494\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") "
Sep 09 20:54:19.301900 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.301810    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-service-ca\") pod \"c40a6696-c9a7-4897-a094-7f30c33e6494\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") "
Sep 09 20:54:19.301900 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.301861    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-oauth-config\") pod \"c40a6696-c9a7-4897-a094-7f30c33e6494\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") "
Sep 09 20:54:19.301900 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.301892    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-oauth-serving-cert\") pod \"c40a6696-c9a7-4897-a094-7f30c33e6494\" (UID: \"c40a6696-c9a7-4897-a094-7f30c33e6494\") "
Sep 09 20:54:19.302278 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.302246    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-service-ca" (OuterVolumeSpecName: "service-ca") pod "c40a6696-c9a7-4897-a094-7f30c33e6494" (UID: "c40a6696-c9a7-4897-a094-7f30c33e6494"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:19.302398 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.302274    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-console-config" (OuterVolumeSpecName: "console-config") pod "c40a6696-c9a7-4897-a094-7f30c33e6494" (UID: "c40a6696-c9a7-4897-a094-7f30c33e6494"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:19.302474 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.302399    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c40a6696-c9a7-4897-a094-7f30c33e6494" (UID: "c40a6696-c9a7-4897-a094-7f30c33e6494"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:54:19.304093 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.304050    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c40a6696-c9a7-4897-a094-7f30c33e6494" (UID: "c40a6696-c9a7-4897-a094-7f30c33e6494"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:19.304219 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.304183    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c40a6696-c9a7-4897-a094-7f30c33e6494-kube-api-access-pzqpp" (OuterVolumeSpecName: "kube-api-access-pzqpp") pod "c40a6696-c9a7-4897-a094-7f30c33e6494" (UID: "c40a6696-c9a7-4897-a094-7f30c33e6494"). InnerVolumeSpecName "kube-api-access-pzqpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:54:19.304410 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.304370    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c40a6696-c9a7-4897-a094-7f30c33e6494" (UID: "c40a6696-c9a7-4897-a094-7f30c33e6494"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:54:19.403554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.403518    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-console-config\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:19.403554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.403548    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:19.403554 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.403557    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-oauth-config\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:19.403769 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.403566    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c40a6696-c9a7-4897-a094-7f30c33e6494-oauth-serving-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:19.403769 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.403574    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzqpp\" (UniqueName: \"kubernetes.io/projected/c40a6696-c9a7-4897-a094-7f30c33e6494-kube-api-access-pzqpp\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:19.403769 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.403583    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c40a6696-c9a7-4897-a094-7f30c33e6494-console-serving-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:54:19.924387 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.924343    2571 generic.go:356] "Generic (PLEG): container finished" podID="c40a6696-c9a7-4897-a094-7f30c33e6494" containerID="8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3" exitCode=0
Sep 09 20:54:19.924581 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.924391    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5fb94fb5-xx9nj"
Sep 09 20:54:19.924581 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.924425    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5fb94fb5-xx9nj" event={"ID":"c40a6696-c9a7-4897-a094-7f30c33e6494","Type":"ContainerDied","Data":"8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3"}
Sep 09 20:54:19.924581 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.924485    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5fb94fb5-xx9nj" event={"ID":"c40a6696-c9a7-4897-a094-7f30c33e6494","Type":"ContainerDied","Data":"497f585a0fc46ae21f3de124c60296492b309b18ca82fc8cb0d467e4f06b97ea"}
Sep 09 20:54:19.924581 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.924507    2571 scope.go:117] "RemoveContainer" containerID="8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3"
Sep 09 20:54:19.933576 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.933559    2571 scope.go:117] "RemoveContainer" containerID="8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3"
Sep 09 20:54:19.933882 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:54:19.933857    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3\": container with ID starting with 8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3 not found: ID does not exist" containerID="8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3"
Sep 09 20:54:19.933982 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.933887    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3"} err="failed to get container status \"8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3\": rpc error: code = NotFound desc = could not find container \"8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3\": container with ID starting with 8c7ebddc9a8910d23887415d2c62bf81418cc4fe28de3392cc987b1fc63ff5a3 not found: ID does not exist"
Sep 09 20:54:19.944787 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.944766    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5fb94fb5-xx9nj"]
Sep 09 20:54:19.948319 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:19.948297    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5fb94fb5-xx9nj"]
Sep 09 20:54:20.565147 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:20.565114    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c40a6696-c9a7-4897-a094-7f30c33e6494" path="/var/lib/kubelet/pods/c40a6696-c9a7-4897-a094-7f30c33e6494/volumes"
Sep 09 20:54:22.743376 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:22.743344    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:54:22.747539 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:22.747512    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78b8c7bc6b-lf6vw"
Sep 09 20:54:24.952299 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:24.952253    2571 generic.go:356] "Generic (PLEG): container finished" podID="54399796-4e0e-4fb9-803a-7d5c20f66b4d" containerID="94c85fc34e3b74ed8bfa16c59ab29d8a935c84639d6e1567b4a1114aa79023b1" exitCode=0
Sep 09 20:54:24.952755 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:24.952335    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z" event={"ID":"54399796-4e0e-4fb9-803a-7d5c20f66b4d","Type":"ContainerDied","Data":"94c85fc34e3b74ed8bfa16c59ab29d8a935c84639d6e1567b4a1114aa79023b1"}
Sep 09 20:54:24.952755 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:24.952745    2571 scope.go:117] "RemoveContainer" containerID="94c85fc34e3b74ed8bfa16c59ab29d8a935c84639d6e1567b4a1114aa79023b1"
Sep 09 20:54:25.959433 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:25.959401    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-dkb8z" event={"ID":"54399796-4e0e-4fb9-803a-7d5c20f66b4d","Type":"ContainerStarted","Data":"acf54280d050f7d9549c7b477abaf928a407c0ffca2a8fc40d46ea86b10c8074"}
Sep 09 20:54:29.976233 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:29.976204    2571 generic.go:356] "Generic (PLEG): container finished" podID="d153405f-bb69-4223-8a81-f3751898f01c" containerID="060476f5fb6ec7ad6bd85fac704e2273908636e49c59fb353f92ce4390e5fa5d" exitCode=0
Sep 09 20:54:29.976597 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:29.976278    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n" event={"ID":"d153405f-bb69-4223-8a81-f3751898f01c","Type":"ContainerDied","Data":"060476f5fb6ec7ad6bd85fac704e2273908636e49c59fb353f92ce4390e5fa5d"}
Sep 09 20:54:29.976597 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:29.976571    2571 scope.go:117] "RemoveContainer" containerID="060476f5fb6ec7ad6bd85fac704e2273908636e49c59fb353f92ce4390e5fa5d"
Sep 09 20:54:30.981993 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:30.981956    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-psg9n" event={"ID":"d153405f-bb69-4223-8a81-f3751898f01c","Type":"ContainerStarted","Data":"b4f060edcc6b4820b9bad7957a86d64b0fa4be6539529f26670b7b4ead751f83"}
Sep 09 20:54:39.479138 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:39.479098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:54:39.481636 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:39.481615    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc-metrics-certs\") pod \"network-metrics-daemon-jffdc\" (UID: \"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc\") " pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:54:39.562813 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:39.562781    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v6qsr\""
Sep 09 20:54:39.570793 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:39.570774    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jffdc"
Sep 09 20:54:39.840670 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:39.840644    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jffdc"]
Sep 09 20:54:39.843466 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:54:39.843432    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd27bdc11_d9f2_4f6f_890d_cf3ec4bf29dc.slice/crio-b40db2d2fc03f466094f7deb933e95cc62ea152f0b2c3198de6e86955171454c WatchSource:0}: Error finding container b40db2d2fc03f466094f7deb933e95cc62ea152f0b2c3198de6e86955171454c: Status 404 returned error can't find the container with id b40db2d2fc03f466094f7deb933e95cc62ea152f0b2c3198de6e86955171454c
Sep 09 20:54:40.018331 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:40.018292    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jffdc" event={"ID":"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc","Type":"ContainerStarted","Data":"b40db2d2fc03f466094f7deb933e95cc62ea152f0b2c3198de6e86955171454c"}
Sep 09 20:54:42.030482 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:42.030445    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jffdc" event={"ID":"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc","Type":"ContainerStarted","Data":"a799ce97d2f69a334c8792ea9a6afbb96cc9a34ea408f33acae980a6670268a1"}
Sep 09 20:54:42.031105 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:42.030507    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jffdc" event={"ID":"d27bdc11-d9f2-4f6f-890d-cf3ec4bf29dc","Type":"ContainerStarted","Data":"595a1d994dd694744efce3b524dede3389c2981cd834edb5d524e21cc4138ff8"}
Sep 09 20:54:42.051203 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:42.051148    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jffdc" podStartSLOduration=373.917453993 podStartE2EDuration="6m15.051131346s" totalImagesPullingTime="1.133677353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:54:42.04931964 +0000 UTC m=+376.019886357" watchObservedRunningTime="2026-09-09 20:54:42.051131346 +0000 UTC m=+376.021698062"
Sep 09 20:54:42.146049 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:42.146011    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:42.204608 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:42.204575    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:49.017852 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:49.017815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:49.077673 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:49.077640    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:57.534130 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:57.534071    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:54:57.730219 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:54:57.730180    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:00.235012 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:00.234975    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:00.299882 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:00.299842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:02.624133 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624100    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-878c97446-lfkw9"]
Sep 09 20:55:02.624502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624431    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58dcb4e-1478-4c22-9ba1-10bd526d6946" containerName="console"
Sep 09 20:55:02.624502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624442    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58dcb4e-1478-4c22-9ba1-10bd526d6946" containerName="console"
Sep 09 20:55:02.624502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624451    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c40a6696-c9a7-4897-a094-7f30c33e6494" containerName="console"
Sep 09 20:55:02.624502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624456    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c40a6696-c9a7-4897-a094-7f30c33e6494" containerName="console"
Sep 09 20:55:02.624502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624466    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6dd829-83b3-4d95-9ee8-241bcdc87792" containerName="registry"
Sep 09 20:55:02.624502 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624471    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6dd829-83b3-4d95-9ee8-241bcdc87792" containerName="registry"
Sep 09 20:55:02.624689 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624531    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6dd829-83b3-4d95-9ee8-241bcdc87792" containerName="registry"
Sep 09 20:55:02.624689 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624539    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58dcb4e-1478-4c22-9ba1-10bd526d6946" containerName="console"
Sep 09 20:55:02.624689 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.624546    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c40a6696-c9a7-4897-a094-7f30c33e6494" containerName="console"
Sep 09 20:55:02.629068 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.629048    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.631246 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.631221    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 09 20:55:02.631835 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.631816    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 09 20:55:02.631967 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.631892    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 09 20:55:02.632480 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.632460    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 09 20:55:02.632818 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.632803    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 09 20:55:02.632895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.632879    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-z5xj4\""
Sep 09 20:55:02.637511 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.637492    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 09 20:55:02.642982 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.642960    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-878c97446-lfkw9"]
Sep 09 20:55:02.779803 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.779764    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.779803 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.779811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-federate-client-tls\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.780017 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.779838    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-secret-telemeter-client\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.780017 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.779898    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-telemeter-client-tls\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.780017 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.779966    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-serving-certs-ca-bundle\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.780017 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.779987    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pwnd\" (UniqueName: \"kubernetes.io/projected/0937c9db-c75d-49f4-81f8-f670bee27b5f-kube-api-access-6pwnd\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.780174 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.780037    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.780174 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.780101    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-metrics-client-ca\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.880829 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.880736    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-metrics-client-ca\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.880829 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.880791    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.880829 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.880823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-federate-client-tls\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.881124 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.880862    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-secret-telemeter-client\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.881124 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.880895    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-telemeter-client-tls\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.881124 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.880951    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-serving-certs-ca-bundle\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.881124 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.880974    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pwnd\" (UniqueName: \"kubernetes.io/projected/0937c9db-c75d-49f4-81f8-f670bee27b5f-kube-api-access-6pwnd\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.881124 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.881005    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.881749 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.881685    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-serving-certs-ca-bundle\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.881749 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.881732    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.881921 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.881785    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0937c9db-c75d-49f4-81f8-f670bee27b5f-metrics-client-ca\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.883633 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.883614    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-federate-client-tls\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.883777 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.883756    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.883820 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.883780    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-secret-telemeter-client\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.883945 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.883924    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0937c9db-c75d-49f4-81f8-f670bee27b5f-telemeter-client-tls\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.888790 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.888768    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pwnd\" (UniqueName: \"kubernetes.io/projected/0937c9db-c75d-49f4-81f8-f670bee27b5f-kube-api-access-6pwnd\") pod \"telemeter-client-878c97446-lfkw9\" (UID: \"0937c9db-c75d-49f4-81f8-f670bee27b5f\") " pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:02.938888 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:02.938854    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-878c97446-lfkw9"
Sep 09 20:55:03.205997 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:03.205964    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:03.219828 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:03.219799    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-878c97446-lfkw9"]
Sep 09 20:55:03.223055 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:55:03.223032    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0937c9db_c75d_49f4_81f8_f670bee27b5f.slice/crio-053c1c6a0403ff62d537657740f20752a963fa8b5f944978c142c5f2da7cd3f0 WatchSource:0}: Error finding container 053c1c6a0403ff62d537657740f20752a963fa8b5f944978c142c5f2da7cd3f0: Status 404 returned error can't find the container with id 053c1c6a0403ff62d537657740f20752a963fa8b5f944978c142c5f2da7cd3f0
Sep 09 20:55:04.122831 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:04.122792    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-878c97446-lfkw9" event={"ID":"0937c9db-c75d-49f4-81f8-f670bee27b5f","Type":"ContainerStarted","Data":"053c1c6a0403ff62d537657740f20752a963fa8b5f944978c142c5f2da7cd3f0"}
Sep 09 20:55:06.134035 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.133992    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-878c97446-lfkw9" event={"ID":"0937c9db-c75d-49f4-81f8-f670bee27b5f","Type":"ContainerStarted","Data":"150d04c9f7327b912299512475471e468865c077fb4f1758631b2070f676d062"}
Sep 09 20:55:06.134464 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.134120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-878c97446-lfkw9" event={"ID":"0937c9db-c75d-49f4-81f8-f670bee27b5f","Type":"ContainerStarted","Data":"e55f50e57697ae8b568c409a1090f236202ef78d6156a272058f804462ca16e4"}
Sep 09 20:55:06.134464 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.134149    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-878c97446-lfkw9" event={"ID":"0937c9db-c75d-49f4-81f8-f670bee27b5f","Type":"ContainerStarted","Data":"f02cafef005fc780e2032c194b95d78600cb1e2a1f18900a79d1452aeca9d0d5"}
Sep 09 20:55:06.167649 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.167587    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-878c97446-lfkw9" podStartSLOduration=2.184041441 podStartE2EDuration="4.167564462s" totalImagesPullingTime="1.983523021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:06.165276502 +0000 UTC m=+400.135843218" watchObservedRunningTime="2026-09-09 20:55:06.167564462 +0000 UTC m=+400.138131180"
Sep 09 20:55:06.278596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.278559    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d87dd8b8c-7dkx4"]
Sep 09 20:55:06.282209 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.282187    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.284418 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.284386    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 20:55:06.284690 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.284676    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 20:55:06.284776 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.284757    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f5srx\""
Sep 09 20:55:06.285472 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.285446    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 20:55:06.285596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.285447    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 20:55:06.285596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.285568    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 20:55:06.290494 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.290468    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 20:55:06.298038 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.298012    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d87dd8b8c-7dkx4"]
Sep 09 20:55:06.304145 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.304118    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-serving-cert\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.304307 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.304167    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-oauth-config\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.304307 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.304204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-console-config\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.304307 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.304239    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-trusted-ca-bundle\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.304307 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.304262    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-oauth-serving-cert\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.304307 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.304291    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-service-ca\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.304510 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.304323    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwss4\" (UniqueName: \"kubernetes.io/projected/e50f3700-5dda-4b9d-aa99-69d736a00085-kube-api-access-bwss4\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.331481 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.331427    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:06.392167 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.392058    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:06.404748 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.404722    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-serving-cert\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.404846 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.404759    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-oauth-config\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.404846 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.404796    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-console-config\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.404846 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.404831    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-trusted-ca-bundle\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.404984 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.404852    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-oauth-serving-cert\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.404984 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.404882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-service-ca\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.404984 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.404918    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwss4\" (UniqueName: \"kubernetes.io/projected/e50f3700-5dda-4b9d-aa99-69d736a00085-kube-api-access-bwss4\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.405716 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.405693    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-trusted-ca-bundle\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.405826 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.405721    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-service-ca\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.405893 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.405827    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-console-config\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.405989 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.405967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-oauth-serving-cert\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.407664 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.407644    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-oauth-config\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.407721 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.407644    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-serving-cert\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.416656 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.416633    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwss4\" (UniqueName: \"kubernetes.io/projected/e50f3700-5dda-4b9d-aa99-69d736a00085-kube-api-access-bwss4\") pod \"console-6d87dd8b8c-7dkx4\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") " pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.591955 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.591920    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:06.864739 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:06.864713    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d87dd8b8c-7dkx4"]
Sep 09 20:55:06.867096 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:55:06.867050    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50f3700_5dda_4b9d_aa99_69d736a00085.slice/crio-82260a1e3183a1bd7536e65d8de14a20c0b4bb7e253da74044fe5baf560f1ab0 WatchSource:0}: Error finding container 82260a1e3183a1bd7536e65d8de14a20c0b4bb7e253da74044fe5baf560f1ab0: Status 404 returned error can't find the container with id 82260a1e3183a1bd7536e65d8de14a20c0b4bb7e253da74044fe5baf560f1ab0
Sep 09 20:55:07.140012 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:07.139972    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d87dd8b8c-7dkx4" event={"ID":"e50f3700-5dda-4b9d-aa99-69d736a00085","Type":"ContainerStarted","Data":"275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af"}
Sep 09 20:55:07.140414 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:07.140031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d87dd8b8c-7dkx4" event={"ID":"e50f3700-5dda-4b9d-aa99-69d736a00085","Type":"ContainerStarted","Data":"82260a1e3183a1bd7536e65d8de14a20c0b4bb7e253da74044fe5baf560f1ab0"}
Sep 09 20:55:07.158791 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:07.158685    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d87dd8b8c-7dkx4" podStartSLOduration=1.158665388 podStartE2EDuration="1.158665388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:07.158244262 +0000 UTC m=+401.128810980" watchObservedRunningTime="2026-09-09 20:55:07.158665388 +0000 UTC m=+401.129232110"
Sep 09 20:55:12.739268 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:12.739226    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:12.841135 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:12.841065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:16.008270 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.008237    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d87dd8b8c-7dkx4"]
Sep 09 20:55:16.041241 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.041206    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b58d59f87-5mg64"]
Sep 09 20:55:16.048313 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.048281    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.059542 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.059514    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b58d59f87-5mg64"]
Sep 09 20:55:16.086264 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.086219    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-serving-cert\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.086431 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.086270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-trusted-ca-bundle\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.086431 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.086343    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-oauth-serving-cert\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.086431 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.086370    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-config\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.086431 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.086386    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-service-ca\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.086583 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.086432    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-oauth-config\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.086583 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.086530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68lfd\" (UniqueName: \"kubernetes.io/projected/07950fbb-b49f-4da1-b9e6-27bc767158a8-kube-api-access-68lfd\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.187415 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.187379    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68lfd\" (UniqueName: \"kubernetes.io/projected/07950fbb-b49f-4da1-b9e6-27bc767158a8-kube-api-access-68lfd\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.187584 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.187436    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-serving-cert\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.187633 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.187613    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-trusted-ca-bundle\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.187668 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.187656    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-oauth-serving-cert\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.187725 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.187704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-config\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.187765 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.187740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-service-ca\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.187798 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.187790    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-oauth-config\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.188450 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.188423    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-oauth-serving-cert\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.188571 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.188525    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-service-ca\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.188799 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.188631    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-trusted-ca-bundle\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.188799 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.188666    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-config\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.190186 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.190167    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-serving-cert\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.190398 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.190382    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-oauth-config\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.206176 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.206149    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68lfd\" (UniqueName: \"kubernetes.io/projected/07950fbb-b49f-4da1-b9e6-27bc767158a8-kube-api-access-68lfd\") pod \"console-6b58d59f87-5mg64\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") " pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.357881 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.357775    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:16.592109 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.592057    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:16.632982 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:16.632957    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b58d59f87-5mg64"]
Sep 09 20:55:16.635455 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:55:16.635428    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07950fbb_b49f_4da1_b9e6_27bc767158a8.slice/crio-61d3bb19cdfb4cd9b5aaa68bc38ea260cd44be6b20ab018f228822f432e0a96a WatchSource:0}: Error finding container 61d3bb19cdfb4cd9b5aaa68bc38ea260cd44be6b20ab018f228822f432e0a96a: Status 404 returned error can't find the container with id 61d3bb19cdfb4cd9b5aaa68bc38ea260cd44be6b20ab018f228822f432e0a96a
Sep 09 20:55:17.183287 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:17.183258    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b58d59f87-5mg64" event={"ID":"07950fbb-b49f-4da1-b9e6-27bc767158a8","Type":"ContainerStarted","Data":"ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf"}
Sep 09 20:55:17.183688 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:17.183304    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b58d59f87-5mg64" event={"ID":"07950fbb-b49f-4da1-b9e6-27bc767158a8","Type":"ContainerStarted","Data":"61d3bb19cdfb4cd9b5aaa68bc38ea260cd44be6b20ab018f228822f432e0a96a"}
Sep 09 20:55:17.203895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:17.203847    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b58d59f87-5mg64" podStartSLOduration=1.20383159 podStartE2EDuration="1.20383159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:55:17.203146773 +0000 UTC m=+411.173713491" watchObservedRunningTime="2026-09-09 20:55:17.20383159 +0000 UTC m=+411.174398307"
Sep 09 20:55:24.761503 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:24.761472    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:25.847398 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:25.847362    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:26.358156 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:26.358107    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:26.358156 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:26.358170    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:26.363013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:26.362991    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:27.234408 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:27.234383    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:55:27.422313 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:27.422249    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:27.527050 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:27.526967    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:27.637880 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:27.637843    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:30.732377 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:30.732342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:30.823190 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:30.823147    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:40.813924 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:40.813885    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:40.890306 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:40.890267    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:55:41.028257 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.028211    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d87dd8b8c-7dkx4" podUID="e50f3700-5dda-4b9d-aa99-69d736a00085" containerName="console" containerID="cri-o://275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af" gracePeriod=15
Sep 09 20:55:41.276250 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.276221    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:41.297718 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.297683    2571 generic.go:356] "Generic (PLEG): container finished" podID="e50f3700-5dda-4b9d-aa99-69d736a00085" containerID="275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af" exitCode=0
Sep 09 20:55:41.297718 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.297729    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d87dd8b8c-7dkx4" event={"ID":"e50f3700-5dda-4b9d-aa99-69d736a00085","Type":"ContainerDied","Data":"275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af"}
Sep 09 20:55:41.298010 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.297775    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d87dd8b8c-7dkx4" event={"ID":"e50f3700-5dda-4b9d-aa99-69d736a00085","Type":"ContainerDied","Data":"82260a1e3183a1bd7536e65d8de14a20c0b4bb7e253da74044fe5baf560f1ab0"}
Sep 09 20:55:41.298010 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.297801    2571 scope.go:117] "RemoveContainer" containerID="275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af"
Sep 09 20:55:41.298010 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.297950    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d87dd8b8c-7dkx4"
Sep 09 20:55:41.307306 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.307284    2571 scope.go:117] "RemoveContainer" containerID="275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af"
Sep 09 20:55:41.307612 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:55:41.307591    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af\": container with ID starting with 275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af not found: ID does not exist" containerID="275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af"
Sep 09 20:55:41.307661 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.307624    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af"} err="failed to get container status \"275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af\": rpc error: code = NotFound desc = could not find container \"275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af\": container with ID starting with 275dd2dc1f667092e51a755a343e495e5913b8a999d4f82efdc14eba209dc3af not found: ID does not exist"
Sep 09 20:55:41.417143 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417038    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-serving-cert\") pod \"e50f3700-5dda-4b9d-aa99-69d736a00085\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") "
Sep 09 20:55:41.417143 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417073    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-service-ca\") pod \"e50f3700-5dda-4b9d-aa99-69d736a00085\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") "
Sep 09 20:55:41.417143 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417120    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-trusted-ca-bundle\") pod \"e50f3700-5dda-4b9d-aa99-69d736a00085\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") "
Sep 09 20:55:41.417415 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417170    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-oauth-serving-cert\") pod \"e50f3700-5dda-4b9d-aa99-69d736a00085\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") "
Sep 09 20:55:41.417415 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417211    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwss4\" (UniqueName: \"kubernetes.io/projected/e50f3700-5dda-4b9d-aa99-69d736a00085-kube-api-access-bwss4\") pod \"e50f3700-5dda-4b9d-aa99-69d736a00085\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") "
Sep 09 20:55:41.417415 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417235    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-oauth-config\") pod \"e50f3700-5dda-4b9d-aa99-69d736a00085\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") "
Sep 09 20:55:41.417415 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417342    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-console-config\") pod \"e50f3700-5dda-4b9d-aa99-69d736a00085\" (UID: \"e50f3700-5dda-4b9d-aa99-69d736a00085\") "
Sep 09 20:55:41.417617 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417571    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-service-ca" (OuterVolumeSpecName: "service-ca") pod "e50f3700-5dda-4b9d-aa99-69d736a00085" (UID: "e50f3700-5dda-4b9d-aa99-69d736a00085"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:41.417669 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417607    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e50f3700-5dda-4b9d-aa99-69d736a00085" (UID: "e50f3700-5dda-4b9d-aa99-69d736a00085"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:41.417669 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417616    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e50f3700-5dda-4b9d-aa99-69d736a00085" (UID: "e50f3700-5dda-4b9d-aa99-69d736a00085"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:41.417806 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417711    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:55:41.417806 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417729    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-trusted-ca-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:55:41.417886 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.417813    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-oauth-serving-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:55:41.418122 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.418098    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-console-config" (OuterVolumeSpecName: "console-config") pod "e50f3700-5dda-4b9d-aa99-69d736a00085" (UID: "e50f3700-5dda-4b9d-aa99-69d736a00085"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:55:41.419472 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.419440    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e50f3700-5dda-4b9d-aa99-69d736a00085" (UID: "e50f3700-5dda-4b9d-aa99-69d736a00085"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:55:41.419576 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.419501    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e50f3700-5dda-4b9d-aa99-69d736a00085" (UID: "e50f3700-5dda-4b9d-aa99-69d736a00085"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:55:41.419576 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.419532    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e50f3700-5dda-4b9d-aa99-69d736a00085-kube-api-access-bwss4" (OuterVolumeSpecName: "kube-api-access-bwss4") pod "e50f3700-5dda-4b9d-aa99-69d736a00085" (UID: "e50f3700-5dda-4b9d-aa99-69d736a00085"). InnerVolumeSpecName "kube-api-access-bwss4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:55:41.518909 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.518861    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-serving-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:55:41.518909 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.518905    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwss4\" (UniqueName: \"kubernetes.io/projected/e50f3700-5dda-4b9d-aa99-69d736a00085-kube-api-access-bwss4\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:55:41.518909 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.518915    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e50f3700-5dda-4b9d-aa99-69d736a00085-console-oauth-config\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:55:41.518909 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.518923    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e50f3700-5dda-4b9d-aa99-69d736a00085-console-config\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:55:41.623410 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.623377    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d87dd8b8c-7dkx4"]
Sep 09 20:55:41.627176 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:41.627148    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d87dd8b8c-7dkx4"]
Sep 09 20:55:42.564973 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:55:42.564938    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e50f3700-5dda-4b9d-aa99-69d736a00085" path="/var/lib/kubelet/pods/e50f3700-5dda-4b9d-aa99-69d736a00085/volumes"
Sep 09 20:56:25.217519 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:25.217473    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:25.349157 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:25.349120    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:26.199166 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.199134    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"]
Sep 09 20:56:26.199483 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.199471    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e50f3700-5dda-4b9d-aa99-69d736a00085" containerName="console"
Sep 09 20:56:26.199524 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.199485    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e50f3700-5dda-4b9d-aa99-69d736a00085" containerName="console"
Sep 09 20:56:26.199565 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.199556    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e50f3700-5dda-4b9d-aa99-69d736a00085" containerName="console"
Sep 09 20:56:26.202706 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.202690    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.205191 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.205170    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:56:26.205995 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.205972    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:56:26.205995 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.205978    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z5ztk\""
Sep 09 20:56:26.212197 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.212174    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"]
Sep 09 20:56:26.291160 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.291114    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z7d4\" (UniqueName: \"kubernetes.io/projected/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-kube-api-access-9z7d4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.291160 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.291154    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.291596 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.291223    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.392555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.392512    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7d4\" (UniqueName: \"kubernetes.io/projected/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-kube-api-access-9z7d4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.392758 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.392564    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.392758 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.392622    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.392978 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.392956    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.393043 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.392996    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.403611 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.403582    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z7d4\" (UniqueName: \"kubernetes.io/projected/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-kube-api-access-9z7d4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.514819 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.514798    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z5ztk\""
Sep 09 20:56:26.522177 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.522152    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:26.794962 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:26.794937    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"]
Sep 09 20:56:26.796982 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:56:26.796954    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c9a74f6_6f8c_4b8c_a4ad_6d4551c836c6.slice/crio-b1c4b325e71f169fb87b7c0685cda1dbd85f7300aeaa83f780873bc49d2333b2 WatchSource:0}: Error finding container b1c4b325e71f169fb87b7c0685cda1dbd85f7300aeaa83f780873bc49d2333b2: Status 404 returned error can't find the container with id b1c4b325e71f169fb87b7c0685cda1dbd85f7300aeaa83f780873bc49d2333b2
Sep 09 20:56:27.509232 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:27.509197    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg" event={"ID":"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6","Type":"ContainerStarted","Data":"b1c4b325e71f169fb87b7c0685cda1dbd85f7300aeaa83f780873bc49d2333b2"}
Sep 09 20:56:33.539402 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:33.539363    2571 generic.go:356] "Generic (PLEG): container finished" podID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerID="29df314abacdb0f54063a2d29f3147a682cfc3b5a4fccd7623dccbb65724513b" exitCode=0
Sep 09 20:56:33.539762 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:33.539439    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg" event={"ID":"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6","Type":"ContainerDied","Data":"29df314abacdb0f54063a2d29f3147a682cfc3b5a4fccd7623dccbb65724513b"}
Sep 09 20:56:36.556555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:36.556517    2571 generic.go:356] "Generic (PLEG): container finished" podID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerID="63eeedf862c495066376de9be5dbe64e7ce2b1eadb20ca0c97e7be9cf1c676e6" exitCode=0
Sep 09 20:56:36.557005 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:36.556600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg" event={"ID":"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6","Type":"ContainerDied","Data":"63eeedf862c495066376de9be5dbe64e7ce2b1eadb20ca0c97e7be9cf1c676e6"}
Sep 09 20:56:43.594358 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:43.594316    2571 generic.go:356] "Generic (PLEG): container finished" podID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerID="6ce9c77e0f539ec70136c1a54dbe4ced544c548885e4a1b202679cf0a65865ae" exitCode=0
Sep 09 20:56:43.594801 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:43.594397    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg" event={"ID":"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6","Type":"ContainerDied","Data":"6ce9c77e0f539ec70136c1a54dbe4ced544c548885e4a1b202679cf0a65865ae"}
Sep 09 20:56:44.726848 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.726823    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:44.769875 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.769849    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z7d4\" (UniqueName: \"kubernetes.io/projected/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-kube-api-access-9z7d4\") pod \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") "
Sep 09 20:56:44.770054 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.769883    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-util\") pod \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") "
Sep 09 20:56:44.770054 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.769911    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-bundle\") pod \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\" (UID: \"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6\") "
Sep 09 20:56:44.770631 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.770557    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-bundle" (OuterVolumeSpecName: "bundle") pod "2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" (UID: "2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:44.772237 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.772211    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-kube-api-access-9z7d4" (OuterVolumeSpecName: "kube-api-access-9z7d4") pod "2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" (UID: "2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6"). InnerVolumeSpecName "kube-api-access-9z7d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:56:44.774044 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.774026    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-util" (OuterVolumeSpecName: "util") pod "2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" (UID: "2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:56:44.871504 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.871420    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z7d4\" (UniqueName: \"kubernetes.io/projected/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-kube-api-access-9z7d4\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:56:44.871504 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.871452    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-util\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:56:44.871504 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:44.871461    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:56:45.605100 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:45.605056    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg" event={"ID":"2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6","Type":"ContainerDied","Data":"b1c4b325e71f169fb87b7c0685cda1dbd85f7300aeaa83f780873bc49d2333b2"}
Sep 09 20:56:45.605276 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:45.605111    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ftlvg"
Sep 09 20:56:45.605276 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:45.605122    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1c4b325e71f169fb87b7c0685cda1dbd85f7300aeaa83f780873bc49d2333b2"
Sep 09 20:56:48.234498 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:48.234450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:48.389810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:48.389781    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:51.523061 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.523027    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"]
Sep 09 20:56:51.523441 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.523407    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerName="pull"
Sep 09 20:56:51.523441 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.523419    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerName="pull"
Sep 09 20:56:51.523441 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.523437    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerName="util"
Sep 09 20:56:51.523441 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.523443    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerName="util"
Sep 09 20:56:51.523565 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.523459    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerName="extract"
Sep 09 20:56:51.523565 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.523464    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerName="extract"
Sep 09 20:56:51.523565 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.523515    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c9a74f6-6f8c-4b8c-a4ad-6d4551c836c6" containerName="extract"
Sep 09 20:56:51.526352 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.526336    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"
Sep 09 20:56:51.544623 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.544590    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:56:51.544766 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.544635    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8n7l8\""
Sep 09 20:56:51.545187 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.545173    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 09 20:56:51.563245 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.563209    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"]
Sep 09 20:56:51.627296 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.627263    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2002d4b5-640d-4f2a-874a-903f23412b53-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2lvwj\" (UID: \"2002d4b5-640d-4f2a-874a-903f23412b53\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"
Sep 09 20:56:51.627468 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.627302    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwwbr\" (UniqueName: \"kubernetes.io/projected/2002d4b5-640d-4f2a-874a-903f23412b53-kube-api-access-hwwbr\") pod \"cert-manager-operator-controller-manager-55c444bf86-2lvwj\" (UID: \"2002d4b5-640d-4f2a-874a-903f23412b53\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"
Sep 09 20:56:51.728619 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.728579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2002d4b5-640d-4f2a-874a-903f23412b53-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2lvwj\" (UID: \"2002d4b5-640d-4f2a-874a-903f23412b53\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"
Sep 09 20:56:51.728619 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.728617    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwwbr\" (UniqueName: \"kubernetes.io/projected/2002d4b5-640d-4f2a-874a-903f23412b53-kube-api-access-hwwbr\") pod \"cert-manager-operator-controller-manager-55c444bf86-2lvwj\" (UID: \"2002d4b5-640d-4f2a-874a-903f23412b53\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"
Sep 09 20:56:51.728971 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.728953    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2002d4b5-640d-4f2a-874a-903f23412b53-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2lvwj\" (UID: \"2002d4b5-640d-4f2a-874a-903f23412b53\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"
Sep 09 20:56:51.759728 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.759698    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwwbr\" (UniqueName: \"kubernetes.io/projected/2002d4b5-640d-4f2a-874a-903f23412b53-kube-api-access-hwwbr\") pod \"cert-manager-operator-controller-manager-55c444bf86-2lvwj\" (UID: \"2002d4b5-640d-4f2a-874a-903f23412b53\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"
Sep 09 20:56:51.835671 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:51.835579    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"
Sep 09 20:56:52.129013 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:52.128935    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj"]
Sep 09 20:56:52.132550 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:56:52.132516    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2002d4b5_640d_4f2a_874a_903f23412b53.slice/crio-7f1f1341e5cd35c1633fd87c8082e613ebb89c2defaae9a2bf3efcb81badeb6f WatchSource:0}: Error finding container 7f1f1341e5cd35c1633fd87c8082e613ebb89c2defaae9a2bf3efcb81badeb6f: Status 404 returned error can't find the container with id 7f1f1341e5cd35c1633fd87c8082e613ebb89c2defaae9a2bf3efcb81badeb6f
Sep 09 20:56:52.640930 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:52.640897    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj" event={"ID":"2002d4b5-640d-4f2a-874a-903f23412b53","Type":"ContainerStarted","Data":"7f1f1341e5cd35c1633fd87c8082e613ebb89c2defaae9a2bf3efcb81badeb6f"}
Sep 09 20:56:53.189733 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:53.189686    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:53.301627 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:53.301593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:54.653423 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:54.653338    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj" event={"ID":"2002d4b5-640d-4f2a-874a-903f23412b53","Type":"ContainerStarted","Data":"d4dd478e1da1338b212bbec0f6967a3e1e11de2ec3de2745362d8d934c6f0882"}
Sep 09 20:56:55.306352 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.306301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:55.564401 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.564299    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:55.710459 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.710413    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2lvwj" podStartSLOduration=2.492569256 podStartE2EDuration="4.710389215s" totalImagesPullingTime="2.217819959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:56:54.693715041 +0000 UTC m=+508.664281758" watchObservedRunningTime="2026-09-09 20:56:55.710389215 +0000 UTC m=+509.680955980"
Sep 09 20:56:55.710861 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.710590    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"]
Sep 09 20:56:55.714318 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.714302    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.717949 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.717930    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z5ztk\""
Sep 09 20:56:55.717949 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.717944    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:56:55.718127 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.717931    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:56:55.734867 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.734840    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"]
Sep 09 20:56:55.767810 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.767776    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.767988 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.767839    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.767988 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.767899    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggwg9\" (UniqueName: \"kubernetes.io/projected/37d619a6-fc32-4263-992c-a6064675b839-kube-api-access-ggwg9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.869047 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.868950    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.869047 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.869023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.869266 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.869056    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggwg9\" (UniqueName: \"kubernetes.io/projected/37d619a6-fc32-4263-992c-a6064675b839-kube-api-access-ggwg9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.869399 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.869379    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.869449 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.869409    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:55.900061 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:55.900027    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggwg9\" (UniqueName: \"kubernetes.io/projected/37d619a6-fc32-4263-992c-a6064675b839-kube-api-access-ggwg9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:56.023492 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.023456    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:56:56.326772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.326745    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"]
Sep 09 20:56:56.329366 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:56:56.329338    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d619a6_fc32_4263_992c_a6064675b839.slice/crio-c9f794edf4ba86e033aac2da52f9cefaae8e9ea8eafaec65dfe39b7e22ef4004 WatchSource:0}: Error finding container c9f794edf4ba86e033aac2da52f9cefaae8e9ea8eafaec65dfe39b7e22ef4004: Status 404 returned error can't find the container with id c9f794edf4ba86e033aac2da52f9cefaae8e9ea8eafaec65dfe39b7e22ef4004
Sep 09 20:56:56.412447 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.412417    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ddvsm"]
Sep 09 20:56:56.415891 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.415875    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:56:56.420612 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.420577    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hppw4\""
Sep 09 20:56:56.420772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.420674    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 20:56:56.428236 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.428216    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 20:56:56.439578 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.439550    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ddvsm"]
Sep 09 20:56:56.473534 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.473503    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc9c4934-9fe8-4292-bbc3-f335822fab5b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ddvsm\" (UID: \"dc9c4934-9fe8-4292-bbc3-f335822fab5b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:56:56.473708 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.473592    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvbj6\" (UniqueName: \"kubernetes.io/projected/dc9c4934-9fe8-4292-bbc3-f335822fab5b-kube-api-access-xvbj6\") pod \"cert-manager-webhook-6486ffd997-ddvsm\" (UID: \"dc9c4934-9fe8-4292-bbc3-f335822fab5b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:56:56.574109 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.574061    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvbj6\" (UniqueName: \"kubernetes.io/projected/dc9c4934-9fe8-4292-bbc3-f335822fab5b-kube-api-access-xvbj6\") pod \"cert-manager-webhook-6486ffd997-ddvsm\" (UID: \"dc9c4934-9fe8-4292-bbc3-f335822fab5b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:56:56.574352 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.574125    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc9c4934-9fe8-4292-bbc3-f335822fab5b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ddvsm\" (UID: \"dc9c4934-9fe8-4292-bbc3-f335822fab5b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:56:56.598003 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.597932    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvbj6\" (UniqueName: \"kubernetes.io/projected/dc9c4934-9fe8-4292-bbc3-f335822fab5b-kube-api-access-xvbj6\") pod \"cert-manager-webhook-6486ffd997-ddvsm\" (UID: \"dc9c4934-9fe8-4292-bbc3-f335822fab5b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:56:56.605471 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.605439    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc9c4934-9fe8-4292-bbc3-f335822fab5b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ddvsm\" (UID: \"dc9c4934-9fe8-4292-bbc3-f335822fab5b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:56:56.664551 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.664516    2571 generic.go:356] "Generic (PLEG): container finished" podID="37d619a6-fc32-4263-992c-a6064675b839" containerID="e6d70c4ac15bcff947e46decdaf5ca90a1aaa35b9acaea09466df1d32b64c7ef" exitCode=0
Sep 09 20:56:56.664727 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.664552    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd" event={"ID":"37d619a6-fc32-4263-992c-a6064675b839","Type":"ContainerDied","Data":"e6d70c4ac15bcff947e46decdaf5ca90a1aaa35b9acaea09466df1d32b64c7ef"}
Sep 09 20:56:56.664727 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.664601    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd" event={"ID":"37d619a6-fc32-4263-992c-a6064675b839","Type":"ContainerStarted","Data":"c9f794edf4ba86e033aac2da52f9cefaae8e9ea8eafaec65dfe39b7e22ef4004"}
Sep 09 20:56:56.740480 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:56.740438    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:56:57.014757 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:57.014730    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ddvsm"]
Sep 09 20:56:57.016985 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:56:57.016955    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc9c4934_9fe8_4292_bbc3_f335822fab5b.slice/crio-76f260650cd5f5945f39c3e30f472658c050b01d905099b9192781340b5aef23 WatchSource:0}: Error finding container 76f260650cd5f5945f39c3e30f472658c050b01d905099b9192781340b5aef23: Status 404 returned error can't find the container with id 76f260650cd5f5945f39c3e30f472658c050b01d905099b9192781340b5aef23
Sep 09 20:56:57.381132 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:57.381100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:57.670022 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:57.669945    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm" event={"ID":"dc9c4934-9fe8-4292-bbc3-f335822fab5b","Type":"ContainerStarted","Data":"76f260650cd5f5945f39c3e30f472658c050b01d905099b9192781340b5aef23"}
Sep 09 20:56:58.010909 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:58.010828    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:59.680613 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:59.680561    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:56:59.784250 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:56:59.784170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:00.688621 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:00.688523    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm" event={"ID":"dc9c4934-9fe8-4292-bbc3-f335822fab5b","Type":"ContainerStarted","Data":"323c17110f4871aed119a79c1293e2a990b08b3baed13620c7388f7578cb25fa"}
Sep 09 20:57:00.689033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:00.688670    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:57:00.690389 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:00.690368    2571 generic.go:356] "Generic (PLEG): container finished" podID="37d619a6-fc32-4263-992c-a6064675b839" containerID="0f9d586a9a6bf22fc772a97bc60eff9c28dc2e948e463a490deae547950a63fb" exitCode=0
Sep 09 20:57:00.690474 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:00.690450    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd" event={"ID":"37d619a6-fc32-4263-992c-a6064675b839","Type":"ContainerDied","Data":"0f9d586a9a6bf22fc772a97bc60eff9c28dc2e948e463a490deae547950a63fb"}
Sep 09 20:57:00.708123 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:00.708060    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm" podStartSLOduration=1.347974179 podStartE2EDuration="4.708043744s" totalImagesPullingTime="3.360069565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:56:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:00.706510979 +0000 UTC m=+514.677077709" watchObservedRunningTime="2026-09-09 20:57:00.708043744 +0000 UTC m=+514.678610461"
Sep 09 20:57:01.697476 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:01.697433    2571 generic.go:356] "Generic (PLEG): container finished" podID="37d619a6-fc32-4263-992c-a6064675b839" containerID="62f1305dee2186576035ffd193d01f4b43abae6a65ba33157cd6014c8e075578" exitCode=0
Sep 09 20:57:01.697842 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:01.697512    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd" event={"ID":"37d619a6-fc32-4263-992c-a6064675b839","Type":"ContainerDied","Data":"62f1305dee2186576035ffd193d01f4b43abae6a65ba33157cd6014c8e075578"}
Sep 09 20:57:02.831230 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:02.831207    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:57:02.934447 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:02.934412    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-util\") pod \"37d619a6-fc32-4263-992c-a6064675b839\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") "
Sep 09 20:57:02.934616 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:02.934461    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggwg9\" (UniqueName: \"kubernetes.io/projected/37d619a6-fc32-4263-992c-a6064675b839-kube-api-access-ggwg9\") pod \"37d619a6-fc32-4263-992c-a6064675b839\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") "
Sep 09 20:57:02.934616 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:02.934541    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-bundle\") pod \"37d619a6-fc32-4263-992c-a6064675b839\" (UID: \"37d619a6-fc32-4263-992c-a6064675b839\") "
Sep 09 20:57:02.934960 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:02.934938    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-bundle" (OuterVolumeSpecName: "bundle") pod "37d619a6-fc32-4263-992c-a6064675b839" (UID: "37d619a6-fc32-4263-992c-a6064675b839"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:02.936745 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:02.936722    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37d619a6-fc32-4263-992c-a6064675b839-kube-api-access-ggwg9" (OuterVolumeSpecName: "kube-api-access-ggwg9") pod "37d619a6-fc32-4263-992c-a6064675b839" (UID: "37d619a6-fc32-4263-992c-a6064675b839"). InnerVolumeSpecName "kube-api-access-ggwg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:57:02.938588 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:02.938552    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-util" (OuterVolumeSpecName: "util") pod "37d619a6-fc32-4263-992c-a6064675b839" (UID: "37d619a6-fc32-4263-992c-a6064675b839"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:03.035312 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.035219    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:03.035312 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.035249    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37d619a6-fc32-4263-992c-a6064675b839-util\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:03.035312 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.035259    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggwg9\" (UniqueName: \"kubernetes.io/projected/37d619a6-fc32-4263-992c-a6064675b839-kube-api-access-ggwg9\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:03.517206 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.517162    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"]
Sep 09 20:57:03.517567 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.517554    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37d619a6-fc32-4263-992c-a6064675b839" containerName="pull"
Sep 09 20:57:03.517620 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.517568    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37d619a6-fc32-4263-992c-a6064675b839" containerName="pull"
Sep 09 20:57:03.517620 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.517580    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37d619a6-fc32-4263-992c-a6064675b839" containerName="util"
Sep 09 20:57:03.517620 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.517585    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37d619a6-fc32-4263-992c-a6064675b839" containerName="util"
Sep 09 20:57:03.517620 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.517595    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37d619a6-fc32-4263-992c-a6064675b839" containerName="extract"
Sep 09 20:57:03.517620 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.517600    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37d619a6-fc32-4263-992c-a6064675b839" containerName="extract"
Sep 09 20:57:03.517795 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.517652    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="37d619a6-fc32-4263-992c-a6064675b839" containerName="extract"
Sep 09 20:57:03.521273 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.521241    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"
Sep 09 20:57:03.523770 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.523744    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-n4kdc\""
Sep 09 20:57:03.529600 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.529573    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"]
Sep 09 20:57:03.640597 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.640554    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvlp4\" (UniqueName: \"kubernetes.io/projected/4a851477-760e-4b06-a8b3-c2e22765b1c7-kube-api-access-dvlp4\") pod \"cert-manager-cainjector-559cdd8c87-h79lc\" (UID: \"4a851477-760e-4b06-a8b3-c2e22765b1c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"
Sep 09 20:57:03.640597 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.640601    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a851477-760e-4b06-a8b3-c2e22765b1c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h79lc\" (UID: \"4a851477-760e-4b06-a8b3-c2e22765b1c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"
Sep 09 20:57:03.709168 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.709133    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd" event={"ID":"37d619a6-fc32-4263-992c-a6064675b839","Type":"ContainerDied","Data":"c9f794edf4ba86e033aac2da52f9cefaae8e9ea8eafaec65dfe39b7e22ef4004"}
Sep 09 20:57:03.709348 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.709174    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flzgmd"
Sep 09 20:57:03.709348 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.709179    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9f794edf4ba86e033aac2da52f9cefaae8e9ea8eafaec65dfe39b7e22ef4004"
Sep 09 20:57:03.741430 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.741389    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvlp4\" (UniqueName: \"kubernetes.io/projected/4a851477-760e-4b06-a8b3-c2e22765b1c7-kube-api-access-dvlp4\") pod \"cert-manager-cainjector-559cdd8c87-h79lc\" (UID: \"4a851477-760e-4b06-a8b3-c2e22765b1c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"
Sep 09 20:57:03.741430 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.741436    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a851477-760e-4b06-a8b3-c2e22765b1c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h79lc\" (UID: \"4a851477-760e-4b06-a8b3-c2e22765b1c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"
Sep 09 20:57:03.749465 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.749440    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a851477-760e-4b06-a8b3-c2e22765b1c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h79lc\" (UID: \"4a851477-760e-4b06-a8b3-c2e22765b1c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"
Sep 09 20:57:03.749609 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.749592    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvlp4\" (UniqueName: \"kubernetes.io/projected/4a851477-760e-4b06-a8b3-c2e22765b1c7-kube-api-access-dvlp4\") pod \"cert-manager-cainjector-559cdd8c87-h79lc\" (UID: \"4a851477-760e-4b06-a8b3-c2e22765b1c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"
Sep 09 20:57:03.832069 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:03.831982    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"
Sep 09 20:57:04.167026 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:04.167000    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h79lc"]
Sep 09 20:57:04.168636 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:57:04.168610    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a851477_760e_4b06_a8b3_c2e22765b1c7.slice/crio-b4bda5feebb94475ae99399416a84d718e7b160c8a7792ab6d88ff58789e6551 WatchSource:0}: Error finding container b4bda5feebb94475ae99399416a84d718e7b160c8a7792ab6d88ff58789e6551: Status 404 returned error can't find the container with id b4bda5feebb94475ae99399416a84d718e7b160c8a7792ab6d88ff58789e6551
Sep 09 20:57:04.715496 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:04.715461    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc" event={"ID":"4a851477-760e-4b06-a8b3-c2e22765b1c7","Type":"ContainerStarted","Data":"0eccd537fa5426304ad6e10f9794109e478fd9dfaa2c32548fdac867d2252007"}
Sep 09 20:57:04.715496 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:04.715510    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc" event={"ID":"4a851477-760e-4b06-a8b3-c2e22765b1c7","Type":"ContainerStarted","Data":"b4bda5feebb94475ae99399416a84d718e7b160c8a7792ab6d88ff58789e6551"}
Sep 09 20:57:04.731252 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:04.731194    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h79lc" podStartSLOduration=1.7311780140000002 podStartE2EDuration="1.731178014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:04.730170432 +0000 UTC m=+518.700737150" watchObservedRunningTime="2026-09-09 20:57:04.731178014 +0000 UTC m=+518.701744730"
Sep 09 20:57:05.020854 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:05.020757    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:05.116634 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:05.116596    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:05.958395 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:05.958360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:06.612697 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:06.612660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:06.699861 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:06.699832    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ddvsm"
Sep 09 20:57:06.712898 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:06.712865    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:12.243348 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:12.243309    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:12.344061 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:12.344022    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:14.078720 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.078690    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:14.174971 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.174934    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"]
Sep 09 20:57:14.182123 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.182063    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.184926 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.184900    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z5ztk\""
Sep 09 20:57:14.185214 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.185168    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:57:14.185214 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.185181    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:57:14.186656 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.186636    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"]
Sep 09 20:57:14.332624 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.332536    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p6hd\" (UniqueName: \"kubernetes.io/projected/4289a28d-2105-40b3-ba17-96ee21942424-kube-api-access-5p6hd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.332624 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.332585    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.332832 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.332652    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.362882 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.362848    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:14.433616 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.433577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p6hd\" (UniqueName: \"kubernetes.io/projected/4289a28d-2105-40b3-ba17-96ee21942424-kube-api-access-5p6hd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.433815 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.433632    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.433815 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.433665    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.434149 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.434126    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.434229 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.434129    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.442463 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.442430    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p6hd\" (UniqueName: \"kubernetes.io/projected/4289a28d-2105-40b3-ba17-96ee21942424-kube-api-access-5p6hd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.492471 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.492439    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:14.764035 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:14.764012    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"]
Sep 09 20:57:14.766406 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:57:14.766378    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4289a28d_2105_40b3_ba17_96ee21942424.slice/crio-91fc2a9515226280f44788532cc22006c0922051af00b320d398d72016099ced WatchSource:0}: Error finding container 91fc2a9515226280f44788532cc22006c0922051af00b320d398d72016099ced: Status 404 returned error can't find the container with id 91fc2a9515226280f44788532cc22006c0922051af00b320d398d72016099ced
Sep 09 20:57:15.769769 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:15.769678    2571 generic.go:356] "Generic (PLEG): container finished" podID="4289a28d-2105-40b3-ba17-96ee21942424" containerID="d4cf201dd83cea8efd73c175dde24f5a445cf8d498a3b36f36d11ff62f5234e5" exitCode=0
Sep 09 20:57:15.770191 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:15.769773    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg" event={"ID":"4289a28d-2105-40b3-ba17-96ee21942424","Type":"ContainerDied","Data":"d4cf201dd83cea8efd73c175dde24f5a445cf8d498a3b36f36d11ff62f5234e5"}
Sep 09 20:57:15.770191 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:15.769866    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg" event={"ID":"4289a28d-2105-40b3-ba17-96ee21942424","Type":"ContainerStarted","Data":"91fc2a9515226280f44788532cc22006c0922051af00b320d398d72016099ced"}
Sep 09 20:57:17.781139 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:17.781030    2571 generic.go:356] "Generic (PLEG): container finished" podID="4289a28d-2105-40b3-ba17-96ee21942424" containerID="e5d66a64cc1cc43baf9a0a69dbaa59d8c6e9c84a7e427747ccdfe2f2f5c966b5" exitCode=0
Sep 09 20:57:17.781139 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:17.781122    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg" event={"ID":"4289a28d-2105-40b3-ba17-96ee21942424","Type":"ContainerDied","Data":"e5d66a64cc1cc43baf9a0a69dbaa59d8c6e9c84a7e427747ccdfe2f2f5c966b5"}
Sep 09 20:57:18.788452 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:18.788416    2571 generic.go:356] "Generic (PLEG): container finished" podID="4289a28d-2105-40b3-ba17-96ee21942424" containerID="995921f7e2f94db1e22cab7826184083a31a36d8c2c0de98cccc783c53dc03e1" exitCode=0
Sep 09 20:57:18.788452 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:18.788452    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg" event={"ID":"4289a28d-2105-40b3-ba17-96ee21942424","Type":"ContainerDied","Data":"995921f7e2f94db1e22cab7826184083a31a36d8c2c0de98cccc783c53dc03e1"}
Sep 09 20:57:19.920728 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:19.920700    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:19.981870 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:19.981828    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p6hd\" (UniqueName: \"kubernetes.io/projected/4289a28d-2105-40b3-ba17-96ee21942424-kube-api-access-5p6hd\") pod \"4289a28d-2105-40b3-ba17-96ee21942424\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") "
Sep 09 20:57:19.982041 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:19.981901    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-util\") pod \"4289a28d-2105-40b3-ba17-96ee21942424\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") "
Sep 09 20:57:19.982041 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:19.982014    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-bundle\") pod \"4289a28d-2105-40b3-ba17-96ee21942424\" (UID: \"4289a28d-2105-40b3-ba17-96ee21942424\") "
Sep 09 20:57:19.982768 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:19.982745    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-bundle" (OuterVolumeSpecName: "bundle") pod "4289a28d-2105-40b3-ba17-96ee21942424" (UID: "4289a28d-2105-40b3-ba17-96ee21942424"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:19.984235 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:19.984212    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4289a28d-2105-40b3-ba17-96ee21942424-kube-api-access-5p6hd" (OuterVolumeSpecName: "kube-api-access-5p6hd") pod "4289a28d-2105-40b3-ba17-96ee21942424" (UID: "4289a28d-2105-40b3-ba17-96ee21942424"). InnerVolumeSpecName "kube-api-access-5p6hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:57:19.987943 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:19.987918    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-util" (OuterVolumeSpecName: "util") pod "4289a28d-2105-40b3-ba17-96ee21942424" (UID: "4289a28d-2105-40b3-ba17-96ee21942424"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:20.083046 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:20.082963    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p6hd\" (UniqueName: \"kubernetes.io/projected/4289a28d-2105-40b3-ba17-96ee21942424-kube-api-access-5p6hd\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:20.083046 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:20.082990    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-util\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:20.083046 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:20.083001    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4289a28d-2105-40b3-ba17-96ee21942424-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:20.664286 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:20.664248    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:20.800253 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:20.800225    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg"
Sep 09 20:57:20.800416 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:20.800257    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjz7cg" event={"ID":"4289a28d-2105-40b3-ba17-96ee21942424","Type":"ContainerDied","Data":"91fc2a9515226280f44788532cc22006c0922051af00b320d398d72016099ced"}
Sep 09 20:57:20.800416 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:20.800330    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91fc2a9515226280f44788532cc22006c0922051af00b320d398d72016099ced"
Sep 09 20:57:24.318033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.318001    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:24.386365 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.386333    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"]
Sep 09 20:57:24.386697 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.386684    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4289a28d-2105-40b3-ba17-96ee21942424" containerName="util"
Sep 09 20:57:24.386744 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.386699    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4289a28d-2105-40b3-ba17-96ee21942424" containerName="util"
Sep 09 20:57:24.386744 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.386716    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4289a28d-2105-40b3-ba17-96ee21942424" containerName="extract"
Sep 09 20:57:24.386744 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.386722    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4289a28d-2105-40b3-ba17-96ee21942424" containerName="extract"
Sep 09 20:57:24.386744 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.386740    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4289a28d-2105-40b3-ba17-96ee21942424" containerName="pull"
Sep 09 20:57:24.386860 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.386745    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4289a28d-2105-40b3-ba17-96ee21942424" containerName="pull"
Sep 09 20:57:24.386860 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.386803    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4289a28d-2105-40b3-ba17-96ee21942424" containerName="extract"
Sep 09 20:57:24.391828 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.391810    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.400068 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.400049    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 20:57:24.400068 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.400059    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 20:57:24.400732 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.400717    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xl7pn\""
Sep 09 20:57:24.400963 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.400948    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 20:57:24.401011 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.400995    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 20:57:24.421686 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.421656    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 20:57:24.435327 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.435299    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"]
Sep 09 20:57:24.481226 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.481191    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:24.523973 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.523933    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4xkf\" (UniqueName: \"kubernetes.io/projected/e3e51597-2888-44c1-882e-1dfbe4beb0da-kube-api-access-f4xkf\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.523973 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.523976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3e51597-2888-44c1-882e-1dfbe4beb0da-metrics-cert\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.524212 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.523995    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3e51597-2888-44c1-882e-1dfbe4beb0da-cert\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.524212 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.524056    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e3e51597-2888-44c1-882e-1dfbe4beb0da-manager-config\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.624705 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.624630    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4xkf\" (UniqueName: \"kubernetes.io/projected/e3e51597-2888-44c1-882e-1dfbe4beb0da-kube-api-access-f4xkf\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.624705 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.624675    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3e51597-2888-44c1-882e-1dfbe4beb0da-metrics-cert\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.624705 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.624698    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3e51597-2888-44c1-882e-1dfbe4beb0da-cert\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.624937 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.624768    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e3e51597-2888-44c1-882e-1dfbe4beb0da-manager-config\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.625601 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.625506    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e3e51597-2888-44c1-882e-1dfbe4beb0da-manager-config\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.627381 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.627363    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3e51597-2888-44c1-882e-1dfbe4beb0da-metrics-cert\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.627561 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.627540    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3e51597-2888-44c1-882e-1dfbe4beb0da-cert\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.633198 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.633172    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4xkf\" (UniqueName: \"kubernetes.io/projected/e3e51597-2888-44c1-882e-1dfbe4beb0da-kube-api-access-f4xkf\") pod \"lws-controller-manager-544fcc9ff7-4tnf7\" (UID: \"e3e51597-2888-44c1-882e-1dfbe4beb0da\") " pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.700697 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.700664    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:24.975157 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:24.975133    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"]
Sep 09 20:57:24.977123 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:57:24.977072    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e51597_2888_44c1_882e_1dfbe4beb0da.slice/crio-aee61c1d597ffc8a5a2b2a80d5732dbabfb780c8f1876d412b704747bb51a7f7 WatchSource:0}: Error finding container aee61c1d597ffc8a5a2b2a80d5732dbabfb780c8f1876d412b704747bb51a7f7: Status 404 returned error can't find the container with id aee61c1d597ffc8a5a2b2a80d5732dbabfb780c8f1876d412b704747bb51a7f7
Sep 09 20:57:25.829133 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:25.829061    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7" event={"ID":"e3e51597-2888-44c1-882e-1dfbe4beb0da","Type":"ContainerStarted","Data":"aee61c1d597ffc8a5a2b2a80d5732dbabfb780c8f1876d412b704747bb51a7f7"}
Sep 09 20:57:26.836133 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:26.836091    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7" event={"ID":"e3e51597-2888-44c1-882e-1dfbe4beb0da","Type":"ContainerStarted","Data":"159ff314295b61cff679a2a3e5936313649d50f6423cafa3fa78048337bdfa9b"}
Sep 09 20:57:26.836511 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:26.836157    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:26.858798 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:26.858736    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7" podStartSLOduration=1.560371312 podStartE2EDuration="2.858714613s" totalImagesPullingTime="1.298343301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:26.854830903 +0000 UTC m=+540.825397623" watchObservedRunningTime="2026-09-09 20:57:26.858714613 +0000 UTC m=+540.829281389"
Sep 09 20:57:28.029410 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.029375    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"]
Sep 09 20:57:28.032654 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.032637    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.035054 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.035029    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:57:28.035192 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.035142    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:57:28.035390 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.035374    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z5ztk\""
Sep 09 20:57:28.043792 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.043768    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"]
Sep 09 20:57:28.158701 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.158662    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.158701 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.158703    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.158911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.158731    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfdt6\" (UniqueName: \"kubernetes.io/projected/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-kube-api-access-hfdt6\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.259772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.259737    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.259772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.259770    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.259972 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.259795    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfdt6\" (UniqueName: \"kubernetes.io/projected/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-kube-api-access-hfdt6\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.260190 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.260157    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.260314 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.260192    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.268299 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.268277    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfdt6\" (UniqueName: \"kubernetes.io/projected/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-kube-api-access-hfdt6\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.342288 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.342184    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:28.641117 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.641073    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"]
Sep 09 20:57:28.642696 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:57:28.642669    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbfdab0_d7d4_4058_8dff_cb6761fdea11.slice/crio-f87fbbf43e7e3022497e29ec206e8317c1463338efdea357d9a66661f971fa6c WatchSource:0}: Error finding container f87fbbf43e7e3022497e29ec206e8317c1463338efdea357d9a66661f971fa6c: Status 404 returned error can't find the container with id f87fbbf43e7e3022497e29ec206e8317c1463338efdea357d9a66661f971fa6c
Sep 09 20:57:28.847171 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.847138    2571 generic.go:356] "Generic (PLEG): container finished" podID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerID="0ec47a1afe4aa73e9e4310ee6765e27bb19a6f839b0c80e56d680fd8ae495235" exitCode=0
Sep 09 20:57:28.847360 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.847213    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p" event={"ID":"1bbfdab0-d7d4-4058-8dff-cb6761fdea11","Type":"ContainerDied","Data":"0ec47a1afe4aa73e9e4310ee6765e27bb19a6f839b0c80e56d680fd8ae495235"}
Sep 09 20:57:28.847360 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:28.847249    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p" event={"ID":"1bbfdab0-d7d4-4058-8dff-cb6761fdea11","Type":"ContainerStarted","Data":"f87fbbf43e7e3022497e29ec206e8317c1463338efdea357d9a66661f971fa6c"}
Sep 09 20:57:29.105807 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:29.105766    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:29.203606 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:29.203569    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:29.679061 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:29.678969    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:29.853903 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:29.853870    2571 generic.go:356] "Generic (PLEG): container finished" podID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerID="205fdf6c0342c13dc18fa515397e756dac2e1493174d7a3e017421322eda3b1e" exitCode=0
Sep 09 20:57:29.854064 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:29.853952    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p" event={"ID":"1bbfdab0-d7d4-4058-8dff-cb6761fdea11","Type":"ContainerDied","Data":"205fdf6c0342c13dc18fa515397e756dac2e1493174d7a3e017421322eda3b1e"}
Sep 09 20:57:29.962453 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:29.962385    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.067693 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.067660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.111095 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.111056    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"]
Sep 09 20:57:30.116841 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.116820    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.120456 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.120420    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 20:57:30.120456 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.120430    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 20:57:30.120456 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.120460    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 20:57:30.120676 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.120449    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 20:57:30.120945 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.120925    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bkhht\""
Sep 09 20:57:30.137690 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.137653    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"]
Sep 09 20:57:30.244279 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.244189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.279850 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.279813    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9925375d-085e-43f6-9067-6938b860b34b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.280008 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.279871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9925375d-085e-43f6-9067-6938b860b34b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.280008 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.279938    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9925375d-085e-43f6-9067-6938b860b34b-manifests\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.280008 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.279967    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6s6m\" (UniqueName: \"kubernetes.io/projected/9925375d-085e-43f6-9067-6938b860b34b-kube-api-access-z6s6m\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.280163 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.280032    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9925375d-085e-43f6-9067-6938b860b34b-tmp\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.343964 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.343926    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.381222 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.381186    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9925375d-085e-43f6-9067-6938b860b34b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.381395 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.381249    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9925375d-085e-43f6-9067-6938b860b34b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.381395 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.381287    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9925375d-085e-43f6-9067-6938b860b34b-manifests\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.381395 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.381316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6s6m\" (UniqueName: \"kubernetes.io/projected/9925375d-085e-43f6-9067-6938b860b34b-kube-api-access-z6s6m\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.381507 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.381392    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9925375d-085e-43f6-9067-6938b860b34b-tmp\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.381781 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.381744    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9925375d-085e-43f6-9067-6938b860b34b-manifests\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.383840 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.383821    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9925375d-085e-43f6-9067-6938b860b34b-tmp\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.384175 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.384158    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9925375d-085e-43f6-9067-6938b860b34b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.384226 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.384202    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9925375d-085e-43f6-9067-6938b860b34b-webhook-cert\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.390304 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.390282    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6s6m\" (UniqueName: \"kubernetes.io/projected/9925375d-085e-43f6-9067-6938b860b34b-kube-api-access-z6s6m\") pod \"opendatahub-operator-controller-manager-d75fb655b-dfcc6\" (UID: \"9925375d-085e-43f6-9067-6938b860b34b\") " pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.428227 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.428186    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:30.444334 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.444302    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.577134 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.577069    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.669635 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.669600    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.740265 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.740243    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"]
Sep 09 20:57:30.741981 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:57:30.741949    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9925375d_085e_43f6_9067_6938b860b34b.slice/crio-b0441fc4fd489dc3100fd7214d3faeb045ee913093179a74a7fe98c65220a84e WatchSource:0}: Error finding container b0441fc4fd489dc3100fd7214d3faeb045ee913093179a74a7fe98c65220a84e: Status 404 returned error can't find the container with id b0441fc4fd489dc3100fd7214d3faeb045ee913093179a74a7fe98c65220a84e
Sep 09 20:57:30.743497 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.743470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:30.860568 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.860484    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6" event={"ID":"9925375d-085e-43f6-9067-6938b860b34b","Type":"ContainerStarted","Data":"b0441fc4fd489dc3100fd7214d3faeb045ee913093179a74a7fe98c65220a84e"}
Sep 09 20:57:30.862622 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.862595    2571 generic.go:356] "Generic (PLEG): container finished" podID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerID="223c627277510ffe9618fed32880ab5c0088104713cb8125632bd2b8c45bef48" exitCode=0
Sep 09 20:57:30.862758 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:30.862685    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p" event={"ID":"1bbfdab0-d7d4-4058-8dff-cb6761fdea11","Type":"ContainerDied","Data":"223c627277510ffe9618fed32880ab5c0088104713cb8125632bd2b8c45bef48"}
Sep 09 20:57:32.047214 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.047186    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:32.203841 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.203803    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-bundle\") pod \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") "
Sep 09 20:57:32.204025 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.203863    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfdt6\" (UniqueName: \"kubernetes.io/projected/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-kube-api-access-hfdt6\") pod \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") "
Sep 09 20:57:32.204025 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.203903    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-util\") pod \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\" (UID: \"1bbfdab0-d7d4-4058-8dff-cb6761fdea11\") "
Sep 09 20:57:32.204834 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.204784    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-bundle" (OuterVolumeSpecName: "bundle") pod "1bbfdab0-d7d4-4058-8dff-cb6761fdea11" (UID: "1bbfdab0-d7d4-4058-8dff-cb6761fdea11"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:32.206686 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.206656    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-kube-api-access-hfdt6" (OuterVolumeSpecName: "kube-api-access-hfdt6") pod "1bbfdab0-d7d4-4058-8dff-cb6761fdea11" (UID: "1bbfdab0-d7d4-4058-8dff-cb6761fdea11"). InnerVolumeSpecName "kube-api-access-hfdt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:57:32.212996 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.212964    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-util" (OuterVolumeSpecName: "util") pod "1bbfdab0-d7d4-4058-8dff-cb6761fdea11" (UID: "1bbfdab0-d7d4-4058-8dff-cb6761fdea11"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:57:32.304965 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.304928    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:32.304965 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.304968    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfdt6\" (UniqueName: \"kubernetes.io/projected/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-kube-api-access-hfdt6\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:32.304965 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.304983    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1bbfdab0-d7d4-4058-8dff-cb6761fdea11-util\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:57:32.879384 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.879305    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p"
Sep 09 20:57:32.879384 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.879328    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08ht7p" event={"ID":"1bbfdab0-d7d4-4058-8dff-cb6761fdea11","Type":"ContainerDied","Data":"f87fbbf43e7e3022497e29ec206e8317c1463338efdea357d9a66661f971fa6c"}
Sep 09 20:57:32.879605 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:32.879387    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f87fbbf43e7e3022497e29ec206e8317c1463338efdea357d9a66661f971fa6c"
Sep 09 20:57:33.887458 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:33.887410    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6" event={"ID":"9925375d-085e-43f6-9067-6938b860b34b","Type":"ContainerStarted","Data":"de7ef10013d3725a2ec02dce0358fd174a5c8c7ddeea0271c8a3875a8265baeb"}
Sep 09 20:57:34.893612 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:34.893583    2571 generic.go:356] "Generic (PLEG): container finished" podID="9925375d-085e-43f6-9067-6938b860b34b" containerID="de7ef10013d3725a2ec02dce0358fd174a5c8c7ddeea0271c8a3875a8265baeb" exitCode=0
Sep 09 20:57:34.893968 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:34.893630    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6" event={"ID":"9925375d-085e-43f6-9067-6938b860b34b","Type":"ContainerDied","Data":"de7ef10013d3725a2ec02dce0358fd174a5c8c7ddeea0271c8a3875a8265baeb"}
Sep 09 20:57:35.902582 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:35.902551    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6" event={"ID":"9925375d-085e-43f6-9067-6938b860b34b","Type":"ContainerStarted","Data":"ac579db153ecfd1d2dc62c883e6e8a6031b7326c91131fc48c8de7f64c06cf4e"}
Sep 09 20:57:35.903125 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:35.902783    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:35.932265 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:35.932209    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6" podStartSLOduration=1.546910075 podStartE2EDuration="5.932193078s" totalImagesPullingTime="3.370736305s" totalInitContainerRuntime="1.014546698s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:57:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:57:35.929175339 +0000 UTC m=+549.899742057" watchObservedRunningTime="2026-09-09 20:57:35.932193078 +0000 UTC m=+549.902759830"
Sep 09 20:57:37.844258 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:37.844223    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-544fcc9ff7-4tnf7"
Sep 09 20:57:46.909281 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:46.909251    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d75fb655b-dfcc6"
Sep 09 20:57:49.868273 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:49.868227    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:57:50.027626 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:57:50.027583    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:01.047159 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.047070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:01.220372 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.220337    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"]
Sep 09 20:58:01.220728 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.220715    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerName="util"
Sep 09 20:58:01.220772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.220729    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerName="util"
Sep 09 20:58:01.220772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.220742    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerName="extract"
Sep 09 20:58:01.220772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.220747    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerName="extract"
Sep 09 20:58:01.220772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.220759    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerName="pull"
Sep 09 20:58:01.220772 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.220765    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerName="pull"
Sep 09 20:58:01.220926 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.220823    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bbfdab0-d7d4-4058-8dff-cb6761fdea11" containerName="extract"
Sep 09 20:58:01.226335 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.226315    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.233242 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.232988    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 20:58:01.233538 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.233512    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 20:58:01.233832 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.233816    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sgvw8\""
Sep 09 20:58:01.233832 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.233855    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 09 20:58:01.234152 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.234135    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 20:58:01.245135 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.245106    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"]
Sep 09 20:58:01.361598 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.361494    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccnkm\" (UniqueName: \"kubernetes.io/projected/6af00cf2-84d6-44ae-9827-2064674cd7df-kube-api-access-ccnkm\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.361598 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.361555    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6af00cf2-84d6-44ae-9827-2064674cd7df-tls-certs\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.361598 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.361594    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af00cf2-84d6-44ae-9827-2064674cd7df-tmp\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.462296 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.462264    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccnkm\" (UniqueName: \"kubernetes.io/projected/6af00cf2-84d6-44ae-9827-2064674cd7df-kube-api-access-ccnkm\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.462494 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.462435    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6af00cf2-84d6-44ae-9827-2064674cd7df-tls-certs\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.462571 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.462536    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af00cf2-84d6-44ae-9827-2064674cd7df-tmp\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.466771 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.465664    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6af00cf2-84d6-44ae-9827-2064674cd7df-tmp\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.466771 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.466709    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6af00cf2-84d6-44ae-9827-2064674cd7df-tls-certs\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.481315 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.481202    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccnkm\" (UniqueName: \"kubernetes.io/projected/6af00cf2-84d6-44ae-9827-2064674cd7df-kube-api-access-ccnkm\") pod \"kube-auth-proxy-6bcf9fffd5-7xzlb\" (UID: \"6af00cf2-84d6-44ae-9827-2064674cd7df\") " pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.536196 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.536156    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"
Sep 09 20:58:01.855134 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:01.855106    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb"]
Sep 09 20:58:01.857236 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:58:01.857210    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af00cf2_84d6_44ae_9827_2064674cd7df.slice/crio-279cd8680d02fa69a088a404b3ae5121d55e37187df413da2a6a935f821403de WatchSource:0}: Error finding container 279cd8680d02fa69a088a404b3ae5121d55e37187df413da2a6a935f821403de: Status 404 returned error can't find the container with id 279cd8680d02fa69a088a404b3ae5121d55e37187df413da2a6a935f821403de
Sep 09 20:58:02.054647 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:02.054614    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb" event={"ID":"6af00cf2-84d6-44ae-9827-2064674cd7df","Type":"ContainerStarted","Data":"279cd8680d02fa69a088a404b3ae5121d55e37187df413da2a6a935f821403de"}
Sep 09 20:58:02.427507 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:02.427271    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:02.879681 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:02.879620    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:03.041583 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:03.041546    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:03.168340 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:03.168247    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:04.629390 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:04.629352    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:05.078700 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:05.078655    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb" event={"ID":"6af00cf2-84d6-44ae-9827-2064674cd7df","Type":"ContainerStarted","Data":"2dc84544febdbf390c8aad79ae79935cdf9a6080bcb0786aa325377d22e82821"}
Sep 09 20:58:05.118498 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:05.118436    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bcf9fffd5-7xzlb" podStartSLOduration=1.100860335 podStartE2EDuration="4.118411144s" totalImagesPullingTime="3.017550809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:05.115193203 +0000 UTC m=+579.085759921" watchObservedRunningTime="2026-09-09 20:58:05.118411144 +0000 UTC m=+579.088977862"
Sep 09 20:58:09.748979 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:09.748929    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:09.967582 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:09.967544    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:10.456679 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:10.456626    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:10.858638 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:10.858535    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:10.971871 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:10.971831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:14.676184 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:14.676133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:14.979951 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:14.979913    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:15.147877 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:15.147829    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:15.287868 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:15.287778    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:15.522704 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:15.522670    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:15.842917 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:15.842872    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:30.775919 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.775883    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8577558f59-5n4zk"]
Sep 09 20:58:30.779352 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.779330    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.796188 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.796154    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8577558f59-5n4zk"]
Sep 09 20:58:30.836221 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.836184    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a8822a-c0d1-4762-a14a-5079f6caa669-console-serving-cert\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.836221 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.836225    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-oauth-serving-cert\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.836426 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.836250    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37a8822a-c0d1-4762-a14a-5079f6caa669-console-oauth-config\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.836426 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.836288    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-trusted-ca-bundle\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.836426 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.836324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-console-config\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.836426 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.836340    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-service-ca\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.836426 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.836355    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djn2x\" (UniqueName: \"kubernetes.io/projected/37a8822a-c0d1-4762-a14a-5079f6caa669-kube-api-access-djn2x\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.855507 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.855476    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:30.937896 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.937858    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a8822a-c0d1-4762-a14a-5079f6caa669-console-serving-cert\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.937896 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.937899    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-oauth-serving-cert\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.938182 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.937921    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37a8822a-c0d1-4762-a14a-5079f6caa669-console-oauth-config\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.938182 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.937935    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-trusted-ca-bundle\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.938182 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.937955    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-console-config\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.938182 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.937975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-service-ca\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.938182 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.937998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djn2x\" (UniqueName: \"kubernetes.io/projected/37a8822a-c0d1-4762-a14a-5079f6caa669-kube-api-access-djn2x\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.938731 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.938707    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-oauth-serving-cert\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.938823 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.938785    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-service-ca\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.938861 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.938842    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-trusted-ca-bundle\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.939017 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.938994    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37a8822a-c0d1-4762-a14a-5079f6caa669-console-config\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.941221 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.941196    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37a8822a-c0d1-4762-a14a-5079f6caa669-console-serving-cert\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.941328 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.941196    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37a8822a-c0d1-4762-a14a-5079f6caa669-console-oauth-config\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:30.947461 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:30.947430    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djn2x\" (UniqueName: \"kubernetes.io/projected/37a8822a-c0d1-4762-a14a-5079f6caa669-kube-api-access-djn2x\") pod \"console-8577558f59-5n4zk\" (UID: \"37a8822a-c0d1-4762-a14a-5079f6caa669\") " pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:31.095687 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:31.095590    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:31.378770 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:31.378744    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8577558f59-5n4zk"]
Sep 09 20:58:31.381054 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:58:31.381022    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a8822a_c0d1_4762_a14a_5079f6caa669.slice/crio-fb2ab5af6a6cd7ff498c27ca9a69a572de6e2a9de9caf42102fa0081a8acd583 WatchSource:0}: Error finding container fb2ab5af6a6cd7ff498c27ca9a69a572de6e2a9de9caf42102fa0081a8acd583: Status 404 returned error can't find the container with id fb2ab5af6a6cd7ff498c27ca9a69a572de6e2a9de9caf42102fa0081a8acd583
Sep 09 20:58:32.221838 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:32.221802    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8577558f59-5n4zk" event={"ID":"37a8822a-c0d1-4762-a14a-5079f6caa669","Type":"ContainerStarted","Data":"2fe885596bbc6cab83ccd4f39cec7fe7e10436b6c726c8460d1e53f6c67839c0"}
Sep 09 20:58:32.222250 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:32.221860    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8577558f59-5n4zk" event={"ID":"37a8822a-c0d1-4762-a14a-5079f6caa669","Type":"ContainerStarted","Data":"fb2ab5af6a6cd7ff498c27ca9a69a572de6e2a9de9caf42102fa0081a8acd583"}
Sep 09 20:58:32.242551 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:32.242502    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8577558f59-5n4zk" podStartSLOduration=2.24248639 podStartE2EDuration="2.24248639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:58:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:58:32.240274397 +0000 UTC m=+606.210841115" watchObservedRunningTime="2026-09-09 20:58:32.24248639 +0000 UTC m=+606.213053107"
Sep 09 20:58:41.096651 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:41.096611    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:41.097071 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:41.096677    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:41.101292 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:41.101273    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:41.269923 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:41.269896    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8577558f59-5n4zk"
Sep 09 20:58:41.324039 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:41.324007    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b58d59f87-5mg64"]
Sep 09 20:58:41.451879 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:41.451847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:41.553140 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:41.553066    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:58:41.653136 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:58:41.653105    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:02.001269 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:02.001180    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:02.098153 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:02.098115    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:05.821016 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:05.820979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:06.173000 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.172915    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"]
Sep 09 20:59:06.176867 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.176848    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.179293 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.179268    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-z5ztk\""
Sep 09 20:59:06.179412 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.179294    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 20:59:06.179412 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.179342    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 20:59:06.183699 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.183677    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"]
Sep 09 20:59:06.261297 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.261260    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5765b\" (UniqueName: \"kubernetes.io/projected/a03c2781-9991-4bd8-9d07-3027fad98fb0-kube-api-access-5765b\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.261472 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.261390    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.261472 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.261434    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.267555 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.267534    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"]
Sep 09 20:59:06.271361 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.271344    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.279774 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.279748    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"]
Sep 09 20:59:06.347234 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.347194    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b58d59f87-5mg64" podUID="07950fbb-b49f-4da1-b9e6-27bc767158a8" containerName="console" containerID="cri-o://ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf" gracePeriod=15
Sep 09 20:59:06.362090 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.362057    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.362200 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.362120    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.362200 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.362148    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wslk5\" (UniqueName: \"kubernetes.io/projected/bad1e245-70ee-4da2-84a5-82fc5679d30c-kube-api-access-wslk5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.362200 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.362186    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.362381 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.362231    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5765b\" (UniqueName: \"kubernetes.io/projected/a03c2781-9991-4bd8-9d07-3027fad98fb0-kube-api-access-5765b\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.362381 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.362324    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.362643 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.362624    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.362702 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.362649    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.369064 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.369038    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"]
Sep 09 20:59:06.371521 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.371495    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5765b\" (UniqueName: \"kubernetes.io/projected/a03c2781-9991-4bd8-9d07-3027fad98fb0-kube-api-access-5765b\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.372974 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.372960    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.380353 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.380331    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"]
Sep 09 20:59:06.463612 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.463564    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.463789 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.463628    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.463789 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.463720    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdkfv\" (UniqueName: \"kubernetes.io/projected/aefca67b-1d63-4d95-92f4-cf13eac3d234-kube-api-access-pdkfv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.463789 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.463755    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.463789 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.463776    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.464004 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.463823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wslk5\" (UniqueName: \"kubernetes.io/projected/bad1e245-70ee-4da2-84a5-82fc5679d30c-kube-api-access-wslk5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.464093 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.464059    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.464156 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.464101    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.470972 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.470943    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"]
Sep 09 20:59:06.475785 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.475765    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.476551 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.476524    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wslk5\" (UniqueName: \"kubernetes.io/projected/bad1e245-70ee-4da2-84a5-82fc5679d30c-kube-api-access-wslk5\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.484055 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.484031    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"]
Sep 09 20:59:06.487884 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.487860    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:06.566033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.564758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.566033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.564805    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g7sf\" (UniqueName: \"kubernetes.io/projected/cc97d010-8d22-46e9-a73a-928a51bf0748-kube-api-access-4g7sf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.566033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.564886    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.566033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.564915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.566033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.564975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.566033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.565007    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdkfv\" (UniqueName: \"kubernetes.io/projected/aefca67b-1d63-4d95-92f4-cf13eac3d234-kube-api-access-pdkfv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.566033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.565707    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.566033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.565972    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.577220 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.577188    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdkfv\" (UniqueName: \"kubernetes.io/projected/aefca67b-1d63-4d95-92f4-cf13eac3d234-kube-api-access-pdkfv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.581900 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.581869    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:06.611358 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.611331    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:59:06.665398 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665373    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68lfd\" (UniqueName: \"kubernetes.io/projected/07950fbb-b49f-4da1-b9e6-27bc767158a8-kube-api-access-68lfd\") pod \"07950fbb-b49f-4da1-b9e6-27bc767158a8\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") "
Sep 09 20:59:06.665553 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665472    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-service-ca\") pod \"07950fbb-b49f-4da1-b9e6-27bc767158a8\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") "
Sep 09 20:59:06.665553 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665502    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-config\") pod \"07950fbb-b49f-4da1-b9e6-27bc767158a8\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") "
Sep 09 20:59:06.665682 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665553    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-trusted-ca-bundle\") pod \"07950fbb-b49f-4da1-b9e6-27bc767158a8\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") "
Sep 09 20:59:06.665682 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665583    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-serving-cert\") pod \"07950fbb-b49f-4da1-b9e6-27bc767158a8\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") "
Sep 09 20:59:06.665682 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665612    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-oauth-serving-cert\") pod \"07950fbb-b49f-4da1-b9e6-27bc767158a8\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") "
Sep 09 20:59:06.665844 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665726    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-oauth-config\") pod \"07950fbb-b49f-4da1-b9e6-27bc767158a8\" (UID: \"07950fbb-b49f-4da1-b9e6-27bc767158a8\") "
Sep 09 20:59:06.665900 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665869    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-service-ca" (OuterVolumeSpecName: "service-ca") pod "07950fbb-b49f-4da1-b9e6-27bc767158a8" (UID: "07950fbb-b49f-4da1-b9e6-27bc767158a8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:06.665900 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665875    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g7sf\" (UniqueName: \"kubernetes.io/projected/cc97d010-8d22-46e9-a73a-928a51bf0748-kube-api-access-4g7sf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.666011 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.665999    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.666126 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.666071    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.666220 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.666170    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.666612 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.666592    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.666762 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.666735    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-config" (OuterVolumeSpecName: "console-config") pod "07950fbb-b49f-4da1-b9e6-27bc767158a8" (UID: "07950fbb-b49f-4da1-b9e6-27bc767158a8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:06.666836 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.666755    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "07950fbb-b49f-4da1-b9e6-27bc767158a8" (UID: "07950fbb-b49f-4da1-b9e6-27bc767158a8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:06.666895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.666870    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "07950fbb-b49f-4da1-b9e6-27bc767158a8" (UID: "07950fbb-b49f-4da1-b9e6-27bc767158a8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 20:59:06.666895 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.666888    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.668754 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.668707    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07950fbb-b49f-4da1-b9e6-27bc767158a8-kube-api-access-68lfd" (OuterVolumeSpecName: "kube-api-access-68lfd") pod "07950fbb-b49f-4da1-b9e6-27bc767158a8" (UID: "07950fbb-b49f-4da1-b9e6-27bc767158a8"). InnerVolumeSpecName "kube-api-access-68lfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:06.669459 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.669432    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "07950fbb-b49f-4da1-b9e6-27bc767158a8" (UID: "07950fbb-b49f-4da1-b9e6-27bc767158a8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:06.670036 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.670001    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "07950fbb-b49f-4da1-b9e6-27bc767158a8" (UID: "07950fbb-b49f-4da1-b9e6-27bc767158a8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 20:59:06.674766 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.674737    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g7sf\" (UniqueName: \"kubernetes.io/projected/cc97d010-8d22-46e9-a73a-928a51bf0748-kube-api-access-4g7sf\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.727852 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.727812    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:06.767408 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.767379    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-oauth-config\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.767408 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.767402    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68lfd\" (UniqueName: \"kubernetes.io/projected/07950fbb-b49f-4da1-b9e6-27bc767158a8-kube-api-access-68lfd\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.767408 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.767414    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-config\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.767645 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.767422    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-trusted-ca-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.767645 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.767430    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07950fbb-b49f-4da1-b9e6-27bc767158a8-console-serving-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.767645 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.767438    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07950fbb-b49f-4da1-b9e6-27bc767158a8-oauth-serving-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:06.780546 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.780359    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"]
Sep 09 20:59:06.783745 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:59:06.783670    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03c2781_9991_4bd8_9d07_3027fad98fb0.slice/crio-d849625c3a5d0aecced89086a0f7f74f8faba9467b54df7830d64830144615c9 WatchSource:0}: Error finding container d849625c3a5d0aecced89086a0f7f74f8faba9467b54df7830d64830144615c9: Status 404 returned error can't find the container with id d849625c3a5d0aecced89086a0f7f74f8faba9467b54df7830d64830144615c9
Sep 09 20:59:06.798970 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.798941    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:06.867546 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:06.867001    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"]
Sep 09 20:59:06.871342 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:59:06.871293    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad1e245_70ee_4da2_84a5_82fc5679d30c.slice/crio-da416ea6fd041ff31ec98271219546ce9e113fa264f267e7dab8d672f9efd5d0 WatchSource:0}: Error finding container da416ea6fd041ff31ec98271219546ce9e113fa264f267e7dab8d672f9efd5d0: Status 404 returned error can't find the container with id da416ea6fd041ff31ec98271219546ce9e113fa264f267e7dab8d672f9efd5d0
Sep 09 20:59:07.005903 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.005876    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"]
Sep 09 20:59:07.031997 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:59:07.031965    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefca67b_1d63_4d95_92f4_cf13eac3d234.slice/crio-55e3e728b1a72c9df0a8d7c555266cde23b7582abc15160dec6a7dd5211206ff WatchSource:0}: Error finding container 55e3e728b1a72c9df0a8d7c555266cde23b7582abc15160dec6a7dd5211206ff: Status 404 returned error can't find the container with id 55e3e728b1a72c9df0a8d7c555266cde23b7582abc15160dec6a7dd5211206ff
Sep 09 20:59:07.078938 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.078915    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"]
Sep 09 20:59:07.085298 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:59:07.085272    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc97d010_8d22_46e9_a73a_928a51bf0748.slice/crio-6fddbc1e155eb35ab51b8e6afe85464322c158ccb41af701806665488ae49749 WatchSource:0}: Error finding container 6fddbc1e155eb35ab51b8e6afe85464322c158ccb41af701806665488ae49749: Status 404 returned error can't find the container with id 6fddbc1e155eb35ab51b8e6afe85464322c158ccb41af701806665488ae49749
Sep 09 20:59:07.406963 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.406928    2571 generic.go:356] "Generic (PLEG): container finished" podID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerID="d2636db2261c15230802ae5a9b0009bcf3ec6936264120d17e8fd88bb6dfdcc6" exitCode=0
Sep 09 20:59:07.407147 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.407026    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq" event={"ID":"cc97d010-8d22-46e9-a73a-928a51bf0748","Type":"ContainerDied","Data":"d2636db2261c15230802ae5a9b0009bcf3ec6936264120d17e8fd88bb6dfdcc6"}
Sep 09 20:59:07.407147 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.407091    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq" event={"ID":"cc97d010-8d22-46e9-a73a-928a51bf0748","Type":"ContainerStarted","Data":"6fddbc1e155eb35ab51b8e6afe85464322c158ccb41af701806665488ae49749"}
Sep 09 20:59:07.411178 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.411159    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 20:59:07.412228 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.412206    2571 generic.go:356] "Generic (PLEG): container finished" podID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerID="61f4e0e1024530a32f129d7c9fe917446c7958bb3e9694704773f1b15e491cd6" exitCode=0
Sep 09 20:59:07.412316 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.412287    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2" event={"ID":"bad1e245-70ee-4da2-84a5-82fc5679d30c","Type":"ContainerDied","Data":"61f4e0e1024530a32f129d7c9fe917446c7958bb3e9694704773f1b15e491cd6"}
Sep 09 20:59:07.412376 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.412331    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2" event={"ID":"bad1e245-70ee-4da2-84a5-82fc5679d30c","Type":"ContainerStarted","Data":"da416ea6fd041ff31ec98271219546ce9e113fa264f267e7dab8d672f9efd5d0"}
Sep 09 20:59:07.413911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.413802    2571 generic.go:356] "Generic (PLEG): container finished" podID="07950fbb-b49f-4da1-b9e6-27bc767158a8" containerID="ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf" exitCode=0
Sep 09 20:59:07.413911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.413842    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b58d59f87-5mg64"
Sep 09 20:59:07.413911 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.413880    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b58d59f87-5mg64" event={"ID":"07950fbb-b49f-4da1-b9e6-27bc767158a8","Type":"ContainerDied","Data":"ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf"}
Sep 09 20:59:07.414106 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.413923    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b58d59f87-5mg64" event={"ID":"07950fbb-b49f-4da1-b9e6-27bc767158a8","Type":"ContainerDied","Data":"61d3bb19cdfb4cd9b5aaa68bc38ea260cd44be6b20ab018f228822f432e0a96a"}
Sep 09 20:59:07.414106 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.413946    2571 scope.go:117] "RemoveContainer" containerID="ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf"
Sep 09 20:59:07.415869 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.415849    2571 generic.go:356] "Generic (PLEG): container finished" podID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerID="fded6da0af795774232337e5d2f3fbf8017a8023c2583c7ed567a8fc453d08ca" exitCode=0
Sep 09 20:59:07.415990 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.415925    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx" event={"ID":"aefca67b-1d63-4d95-92f4-cf13eac3d234","Type":"ContainerDied","Data":"fded6da0af795774232337e5d2f3fbf8017a8023c2583c7ed567a8fc453d08ca"}
Sep 09 20:59:07.415990 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.415972    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx" event={"ID":"aefca67b-1d63-4d95-92f4-cf13eac3d234","Type":"ContainerStarted","Data":"55e3e728b1a72c9df0a8d7c555266cde23b7582abc15160dec6a7dd5211206ff"}
Sep 09 20:59:07.417868 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.417849    2571 generic.go:356] "Generic (PLEG): container finished" podID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerID="e513d3d097ef50972ec06ab6741f3da0b5a37475d1cc3db567eb64c25235348e" exitCode=0
Sep 09 20:59:07.417962 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.417940    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw" event={"ID":"a03c2781-9991-4bd8-9d07-3027fad98fb0","Type":"ContainerDied","Data":"e513d3d097ef50972ec06ab6741f3da0b5a37475d1cc3db567eb64c25235348e"}
Sep 09 20:59:07.418033 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.417969    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw" event={"ID":"a03c2781-9991-4bd8-9d07-3027fad98fb0","Type":"ContainerStarted","Data":"d849625c3a5d0aecced89086a0f7f74f8faba9467b54df7830d64830144615c9"}
Sep 09 20:59:07.424952 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.424921    2571 scope.go:117] "RemoveContainer" containerID="ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf"
Sep 09 20:59:07.425356 ip-10-0-129-13 kubenswrapper[2571]: E0909 20:59:07.425328    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf\": container with ID starting with ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf not found: ID does not exist" containerID="ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf"
Sep 09 20:59:07.425510 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.425365    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf"} err="failed to get container status \"ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf\": rpc error: code = NotFound desc = could not find container \"ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf\": container with ID starting with ce1e8c21e7ab11b09c500e60a1d1d768230477827f2725ee7e212b610ace0ccf not found: ID does not exist"
Sep 09 20:59:07.454058 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.454029    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b58d59f87-5mg64"]
Sep 09 20:59:07.458123 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.458100    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b58d59f87-5mg64"]
Sep 09 20:59:07.663749 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.663650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:07.764050 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:07.764013    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:08.432548 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:08.432521    2571 generic.go:356] "Generic (PLEG): container finished" podID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerID="8987fab56759dab8aeab2db85d05fe0991808909faad13c23a1f43da6278a622" exitCode=0
Sep 09 20:59:08.433007 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:08.432603    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq" event={"ID":"cc97d010-8d22-46e9-a73a-928a51bf0748","Type":"ContainerDied","Data":"8987fab56759dab8aeab2db85d05fe0991808909faad13c23a1f43da6278a622"}
Sep 09 20:59:08.435758 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:08.435711    2571 generic.go:356] "Generic (PLEG): container finished" podID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerID="9a2f2fa0004ca254e80e86cd717b4c5a24c8287e4f328c63cfc8fd9359c270eb" exitCode=0
Sep 09 20:59:08.435832 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:08.435776    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw" event={"ID":"a03c2781-9991-4bd8-9d07-3027fad98fb0","Type":"ContainerDied","Data":"9a2f2fa0004ca254e80e86cd717b4c5a24c8287e4f328c63cfc8fd9359c270eb"}
Sep 09 20:59:08.569881 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:08.569848    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07950fbb-b49f-4da1-b9e6-27bc767158a8" path="/var/lib/kubelet/pods/07950fbb-b49f-4da1-b9e6-27bc767158a8/volumes"
Sep 09 20:59:09.443837 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:09.443810    2571 generic.go:356] "Generic (PLEG): container finished" podID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerID="7a570e2a54bd22e0a3fe7427eaf23a25e52eba6508089c0bd15c745454235796" exitCode=0
Sep 09 20:59:09.444178 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:09.443900    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2" event={"ID":"bad1e245-70ee-4da2-84a5-82fc5679d30c","Type":"ContainerDied","Data":"7a570e2a54bd22e0a3fe7427eaf23a25e52eba6508089c0bd15c745454235796"}
Sep 09 20:59:09.445931 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:09.445907    2571 generic.go:356] "Generic (PLEG): container finished" podID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerID="f24b12aeead306532c7189996ef22bec1c10987ec03c1b4dd9b0d975c5bc1200" exitCode=0
Sep 09 20:59:09.446052 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:09.445985    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx" event={"ID":"aefca67b-1d63-4d95-92f4-cf13eac3d234","Type":"ContainerDied","Data":"f24b12aeead306532c7189996ef22bec1c10987ec03c1b4dd9b0d975c5bc1200"}
Sep 09 20:59:09.448475 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:09.448452    2571 generic.go:356] "Generic (PLEG): container finished" podID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerID="64296fe8c2cf108b717632da071df4ae55ee72d75bef403c37e473b119e3b3b3" exitCode=0
Sep 09 20:59:09.448556 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:09.448522    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw" event={"ID":"a03c2781-9991-4bd8-9d07-3027fad98fb0","Type":"ContainerDied","Data":"64296fe8c2cf108b717632da071df4ae55ee72d75bef403c37e473b119e3b3b3"}
Sep 09 20:59:09.450802 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:09.450782    2571 generic.go:356] "Generic (PLEG): container finished" podID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerID="ed153644b625ccf9f9a91f2affa572880e388668f88d3ec15b4902b80a3df0ad" exitCode=0
Sep 09 20:59:09.450873 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:09.450808    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq" event={"ID":"cc97d010-8d22-46e9-a73a-928a51bf0748","Type":"ContainerDied","Data":"ed153644b625ccf9f9a91f2affa572880e388668f88d3ec15b4902b80a3df0ad"}
Sep 09 20:59:10.458029 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.457994    2571 generic.go:356] "Generic (PLEG): container finished" podID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerID="9e4dbe710fcad3bc80e41da9683c27eeba51d521f0470b0eba498c7b519b0e47" exitCode=0
Sep 09 20:59:10.458505 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.458115    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2" event={"ID":"bad1e245-70ee-4da2-84a5-82fc5679d30c","Type":"ContainerDied","Data":"9e4dbe710fcad3bc80e41da9683c27eeba51d521f0470b0eba498c7b519b0e47"}
Sep 09 20:59:10.460196 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.460171    2571 generic.go:356] "Generic (PLEG): container finished" podID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerID="b21243cac5e934be8310f7797322acb4fb9954644e3f86265ff96c2462160a62" exitCode=0
Sep 09 20:59:10.460323 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.460295    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx" event={"ID":"aefca67b-1d63-4d95-92f4-cf13eac3d234","Type":"ContainerDied","Data":"b21243cac5e934be8310f7797322acb4fb9954644e3f86265ff96c2462160a62"}
Sep 09 20:59:10.596218 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.596194    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:10.633643 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.633619    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:10.707737 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.707702    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-bundle\") pod \"a03c2781-9991-4bd8-9d07-3027fad98fb0\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") "
Sep 09 20:59:10.707907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.707767    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-bundle\") pod \"cc97d010-8d22-46e9-a73a-928a51bf0748\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") "
Sep 09 20:59:10.707907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.707809    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-util\") pod \"cc97d010-8d22-46e9-a73a-928a51bf0748\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") "
Sep 09 20:59:10.707907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.707877    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5765b\" (UniqueName: \"kubernetes.io/projected/a03c2781-9991-4bd8-9d07-3027fad98fb0-kube-api-access-5765b\") pod \"a03c2781-9991-4bd8-9d07-3027fad98fb0\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") "
Sep 09 20:59:10.707907 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.707898    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g7sf\" (UniqueName: \"kubernetes.io/projected/cc97d010-8d22-46e9-a73a-928a51bf0748-kube-api-access-4g7sf\") pod \"cc97d010-8d22-46e9-a73a-928a51bf0748\" (UID: \"cc97d010-8d22-46e9-a73a-928a51bf0748\") "
Sep 09 20:59:10.708191 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.707932    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-util\") pod \"a03c2781-9991-4bd8-9d07-3027fad98fb0\" (UID: \"a03c2781-9991-4bd8-9d07-3027fad98fb0\") "
Sep 09 20:59:10.708488 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.708459    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-bundle" (OuterVolumeSpecName: "bundle") pod "cc97d010-8d22-46e9-a73a-928a51bf0748" (UID: "cc97d010-8d22-46e9-a73a-928a51bf0748"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:10.708657 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.708635    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-bundle" (OuterVolumeSpecName: "bundle") pod "a03c2781-9991-4bd8-9d07-3027fad98fb0" (UID: "a03c2781-9991-4bd8-9d07-3027fad98fb0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:10.710732 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.710710    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc97d010-8d22-46e9-a73a-928a51bf0748-kube-api-access-4g7sf" (OuterVolumeSpecName: "kube-api-access-4g7sf") pod "cc97d010-8d22-46e9-a73a-928a51bf0748" (UID: "cc97d010-8d22-46e9-a73a-928a51bf0748"). InnerVolumeSpecName "kube-api-access-4g7sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:10.710848 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.710734    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a03c2781-9991-4bd8-9d07-3027fad98fb0-kube-api-access-5765b" (OuterVolumeSpecName: "kube-api-access-5765b") pod "a03c2781-9991-4bd8-9d07-3027fad98fb0" (UID: "a03c2781-9991-4bd8-9d07-3027fad98fb0"). InnerVolumeSpecName "kube-api-access-5765b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:10.713632 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.713610    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-util" (OuterVolumeSpecName: "util") pod "a03c2781-9991-4bd8-9d07-3027fad98fb0" (UID: "a03c2781-9991-4bd8-9d07-3027fad98fb0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:10.714685 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.714664    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-util" (OuterVolumeSpecName: "util") pod "cc97d010-8d22-46e9-a73a-928a51bf0748" (UID: "cc97d010-8d22-46e9-a73a-928a51bf0748"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:10.809029 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.808976    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-util\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:10.809029 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.809024    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a03c2781-9991-4bd8-9d07-3027fad98fb0-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:10.809029 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.809035    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:10.809029 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.809042    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc97d010-8d22-46e9-a73a-928a51bf0748-util\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:10.809029 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.809051    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5765b\" (UniqueName: \"kubernetes.io/projected/a03c2781-9991-4bd8-9d07-3027fad98fb0-kube-api-access-5765b\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:10.809362 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:10.809059    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g7sf\" (UniqueName: \"kubernetes.io/projected/cc97d010-8d22-46e9-a73a-928a51bf0748-kube-api-access-4g7sf\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:11.467629 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.467598    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq"
Sep 09 20:59:11.467629 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.467613    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cwslpq" event={"ID":"cc97d010-8d22-46e9-a73a-928a51bf0748","Type":"ContainerDied","Data":"6fddbc1e155eb35ab51b8e6afe85464322c158ccb41af701806665488ae49749"}
Sep 09 20:59:11.468152 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.467667    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6fddbc1e155eb35ab51b8e6afe85464322c158ccb41af701806665488ae49749"
Sep 09 20:59:11.469649 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.469618    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw" event={"ID":"a03c2781-9991-4bd8-9d07-3027fad98fb0","Type":"ContainerDied","Data":"d849625c3a5d0aecced89086a0f7f74f8faba9467b54df7830d64830144615c9"}
Sep 09 20:59:11.469649 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.469649    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d849625c3a5d0aecced89086a0f7f74f8faba9467b54df7830d64830144615c9"
Sep 09 20:59:11.469822 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.469683    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at6sdw"
Sep 09 20:59:11.629275 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.629250    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:11.631317 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.631299    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:11.718655 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.718569    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-util\") pod \"bad1e245-70ee-4da2-84a5-82fc5679d30c\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") "
Sep 09 20:59:11.718655 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.718628    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-bundle\") pod \"aefca67b-1d63-4d95-92f4-cf13eac3d234\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") "
Sep 09 20:59:11.718841 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.718669    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-bundle\") pod \"bad1e245-70ee-4da2-84a5-82fc5679d30c\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") "
Sep 09 20:59:11.718841 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.718724    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wslk5\" (UniqueName: \"kubernetes.io/projected/bad1e245-70ee-4da2-84a5-82fc5679d30c-kube-api-access-wslk5\") pod \"bad1e245-70ee-4da2-84a5-82fc5679d30c\" (UID: \"bad1e245-70ee-4da2-84a5-82fc5679d30c\") "
Sep 09 20:59:11.718841 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.718753    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-util\") pod \"aefca67b-1d63-4d95-92f4-cf13eac3d234\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") "
Sep 09 20:59:11.718841 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.718800    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdkfv\" (UniqueName: \"kubernetes.io/projected/aefca67b-1d63-4d95-92f4-cf13eac3d234-kube-api-access-pdkfv\") pod \"aefca67b-1d63-4d95-92f4-cf13eac3d234\" (UID: \"aefca67b-1d63-4d95-92f4-cf13eac3d234\") "
Sep 09 20:59:11.719260 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.719233    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-bundle" (OuterVolumeSpecName: "bundle") pod "aefca67b-1d63-4d95-92f4-cf13eac3d234" (UID: "aefca67b-1d63-4d95-92f4-cf13eac3d234"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:11.719624 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.719596    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-bundle" (OuterVolumeSpecName: "bundle") pod "bad1e245-70ee-4da2-84a5-82fc5679d30c" (UID: "bad1e245-70ee-4da2-84a5-82fc5679d30c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:11.721255 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.721228    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aefca67b-1d63-4d95-92f4-cf13eac3d234-kube-api-access-pdkfv" (OuterVolumeSpecName: "kube-api-access-pdkfv") pod "aefca67b-1d63-4d95-92f4-cf13eac3d234" (UID: "aefca67b-1d63-4d95-92f4-cf13eac3d234"). InnerVolumeSpecName "kube-api-access-pdkfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:11.721358 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.721232    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad1e245-70ee-4da2-84a5-82fc5679d30c-kube-api-access-wslk5" (OuterVolumeSpecName: "kube-api-access-wslk5") pod "bad1e245-70ee-4da2-84a5-82fc5679d30c" (UID: "bad1e245-70ee-4da2-84a5-82fc5679d30c"). InnerVolumeSpecName "kube-api-access-wslk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 20:59:11.724135 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.724112    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-util" (OuterVolumeSpecName: "util") pod "bad1e245-70ee-4da2-84a5-82fc5679d30c" (UID: "bad1e245-70ee-4da2-84a5-82fc5679d30c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:11.724921 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.724898    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-util" (OuterVolumeSpecName: "util") pod "aefca67b-1d63-4d95-92f4-cf13eac3d234" (UID: "aefca67b-1d63-4d95-92f4-cf13eac3d234"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 20:59:11.820513 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.820455    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:11.820513 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.820501    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-bundle\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:11.820513 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.820510    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wslk5\" (UniqueName: \"kubernetes.io/projected/bad1e245-70ee-4da2-84a5-82fc5679d30c-kube-api-access-wslk5\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:11.820513 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.820520    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aefca67b-1d63-4d95-92f4-cf13eac3d234-util\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:11.820513 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.820529    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdkfv\" (UniqueName: \"kubernetes.io/projected/aefca67b-1d63-4d95-92f4-cf13eac3d234-kube-api-access-pdkfv\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:11.820513 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:11.820536    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bad1e245-70ee-4da2-84a5-82fc5679d30c-util\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 20:59:12.477490 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:12.477458    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2"
Sep 09 20:59:12.477490 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:12.477475    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6zx9p2" event={"ID":"bad1e245-70ee-4da2-84a5-82fc5679d30c","Type":"ContainerDied","Data":"da416ea6fd041ff31ec98271219546ce9e113fa264f267e7dab8d672f9efd5d0"}
Sep 09 20:59:12.477965 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:12.477526    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da416ea6fd041ff31ec98271219546ce9e113fa264f267e7dab8d672f9efd5d0"
Sep 09 20:59:12.479456 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:12.479434    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx"
Sep 09 20:59:12.479582 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:12.479459    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7l8mzx" event={"ID":"aefca67b-1d63-4d95-92f4-cf13eac3d234","Type":"ContainerDied","Data":"55e3e728b1a72c9df0a8d7c555266cde23b7582abc15160dec6a7dd5211206ff"}
Sep 09 20:59:12.479582 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:12.479494    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55e3e728b1a72c9df0a8d7c555266cde23b7582abc15160dec6a7dd5211206ff"
Sep 09 20:59:14.777296 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:14.777264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:17.419260 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:17.419219    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:17.774012 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:17.773979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:18.239934 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:18.239900    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:19.121664 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:19.121626    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:20.896941 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:20.896903    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:21.010591 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:21.010555    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:21.871829 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:21.871780    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:22.290875 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:22.290843    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:22.508532 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:22.508495    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:22.894262 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:22.894221    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:25.040993 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:25.040961    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:26.284961 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:26.284925    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:28.307547 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:28.307505    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:28.500648 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:28.500612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:29.876680 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.876641    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"]
Sep 09 20:59:29.877049 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877037    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerName="pull"
Sep 09 20:59:29.877049 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877048    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerName="pull"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877056    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07950fbb-b49f-4da1-b9e6-27bc767158a8" containerName="console"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877062    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="07950fbb-b49f-4da1-b9e6-27bc767158a8" containerName="console"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877069    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerName="extract"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877099    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerName="extract"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877111    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerName="util"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877116    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerName="util"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877121    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerName="extract"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877125    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerName="extract"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877138    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerName="util"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877143    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerName="util"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877150    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerName="util"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877156    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerName="util"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877161    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerName="pull"
Sep 09 20:59:29.877161 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877165    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerName="pull"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877175    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerName="util"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877180    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerName="util"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877186    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerName="pull"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877190    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerName="pull"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877196    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerName="extract"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877201    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerName="extract"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877208    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerName="extract"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877212    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerName="extract"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877223    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerName="pull"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877227    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerName="pull"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877284    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="07950fbb-b49f-4da1-b9e6-27bc767158a8" containerName="console"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877291    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a03c2781-9991-4bd8-9d07-3027fad98fb0" containerName="extract"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877298    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bad1e245-70ee-4da2-84a5-82fc5679d30c" containerName="extract"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877305    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aefca67b-1d63-4d95-92f4-cf13eac3d234" containerName="extract"
Sep 09 20:59:29.877563 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.877309    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc97d010-8d22-46e9-a73a-928a51bf0748" containerName="extract"
Sep 09 20:59:29.881328 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.881303    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:29.881578 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.881563    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:29.884766 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.884748    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xp86x\""
Sep 09 20:59:29.885027 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.885015    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 20:59:29.885327 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.885315    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 09 20:59:29.885619 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.885601    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 20:59:29.885833 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.885813    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 09 20:59:29.892134 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.892110    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"]
Sep 09 20:59:29.986598 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.986560    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d38f9345-15de-4590-b6cd-faf82fd4c4f3-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:29.986813 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.986609    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8blfp\" (UniqueName: \"kubernetes.io/projected/d38f9345-15de-4590-b6cd-faf82fd4c4f3-kube-api-access-8blfp\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:29.986813 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:29.986741    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d38f9345-15de-4590-b6cd-faf82fd4c4f3-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:30.064365 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.064331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:30.087882 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.087860    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8blfp\" (UniqueName: \"kubernetes.io/projected/d38f9345-15de-4590-b6cd-faf82fd4c4f3-kube-api-access-8blfp\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:30.088002 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.087924    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d38f9345-15de-4590-b6cd-faf82fd4c4f3-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:30.088002 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.087971    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d38f9345-15de-4590-b6cd-faf82fd4c4f3-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:30.088647 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.088631    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d38f9345-15de-4590-b6cd-faf82fd4c4f3-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:30.090490 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.090471    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d38f9345-15de-4590-b6cd-faf82fd4c4f3-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:30.100508 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.100486    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8blfp\" (UniqueName: \"kubernetes.io/projected/d38f9345-15de-4590-b6cd-faf82fd4c4f3-kube-api-access-8blfp\") pod \"kuadrant-console-plugin-fbdd4f445-8222x\" (UID: \"d38f9345-15de-4590-b6cd-faf82fd4c4f3\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:30.192192 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.192112    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"
Sep 09 20:59:30.471289 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.471264    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x"]
Sep 09 20:59:30.472292 ip-10-0-129-13 kubenswrapper[2571]: W0909 20:59:30.472266    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd38f9345_15de_4590_b6cd_faf82fd4c4f3.slice/crio-ba89d0d8a5725638aa6074886bd67df938e5e9a01cf5565e33db8f0992a8bd84 WatchSource:0}: Error finding container ba89d0d8a5725638aa6074886bd67df938e5e9a01cf5565e33db8f0992a8bd84: Status 404 returned error can't find the container with id ba89d0d8a5725638aa6074886bd67df938e5e9a01cf5565e33db8f0992a8bd84
Sep 09 20:59:30.583791 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:30.583755    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x" event={"ID":"d38f9345-15de-4590-b6cd-faf82fd4c4f3","Type":"ContainerStarted","Data":"ba89d0d8a5725638aa6074886bd67df938e5e9a01cf5565e33db8f0992a8bd84"}
Sep 09 20:59:33.806145 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:33.806109    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:33.916227 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:33.916174    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:34.012306 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:34.012266    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:34.115306 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:34.115210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:34.213497 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:34.213453    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:35.617030 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:35.616989    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x" event={"ID":"d38f9345-15de-4590-b6cd-faf82fd4c4f3","Type":"ContainerStarted","Data":"1cf4c24356557957618a648a351851310d6b8faaac8f75040d4bdcb251621a2e"}
Sep 09 20:59:41.753698 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:41.753664    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:41.863512 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:41.863480    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:44.721042 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:44.721001    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:44.824383 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:44.824312    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:47.482153 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:47.482118    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:47.584638 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:47.584601    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:50.064636 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:50.064603    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:50.167622 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:50.167585    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:52.761342 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:52.761302    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:52.860056 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:52.860018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:53.153241 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:53.153151    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:53.251177 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:53.251141    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:53.577843 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:53.577805    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 20:59:53.709028 ip-10-0-129-13 kubenswrapper[2571]: I0909 20:59:53.708964    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:01.582016 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:01.581980    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:01.812158 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:01.812116    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:04.685450 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:04.685418    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:05.164950 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:05.164913    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:11.280304 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:11.280266    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:11.525377 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:11.525338    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:13.726445 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.726384    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8222x" podStartSLOduration=40.402126577 podStartE2EDuration="44.726365514s" totalImagesPullingTime="4.324238937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 20:59:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 20:59:35.658334074 +0000 UTC m=+669.628900791" watchObservedRunningTime="2026-09-09 21:00:13.726365514 +0000 UTC m=+707.696932230"
Sep 09 21:00:13.726850 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.726740    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4gz6w"]
Sep 09 21:00:13.753944 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.753918    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4gz6w"]
Sep 09 21:00:13.754123 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.754031    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:13.758623 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.758599    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 21:00:13.770495 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.770451    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:13.851116 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.851058    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4gz6w"]
Sep 09 21:00:13.886047 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.886014    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-566v2\" (UniqueName: \"kubernetes.io/projected/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-kube-api-access-566v2\") pod \"limitador-limitador-8588749977-4gz6w\" (UID: \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\") " pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:13.886225 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.886057    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-config-file\") pod \"limitador-limitador-8588749977-4gz6w\" (UID: \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\") " pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:13.908693 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.908641    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:13.987142 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.987023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-566v2\" (UniqueName: \"kubernetes.io/projected/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-kube-api-access-566v2\") pod \"limitador-limitador-8588749977-4gz6w\" (UID: \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\") " pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:13.987142 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.987063    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-config-file\") pod \"limitador-limitador-8588749977-4gz6w\" (UID: \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\") " pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:13.987659 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.987641    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-config-file\") pod \"limitador-limitador-8588749977-4gz6w\" (UID: \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\") " pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:13.996828 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:13.996800    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-566v2\" (UniqueName: \"kubernetes.io/projected/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-kube-api-access-566v2\") pod \"limitador-limitador-8588749977-4gz6w\" (UID: \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\") " pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:14.064662 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.064629    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:14.292281 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.292195    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:14.342918 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.342888    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4gz6w"]
Sep 09 21:00:14.344184 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:00:14.344155    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode028d0e2_0b0a_46a1_b6da_e64a09b4c767.slice/crio-ada42bf714f75c66b2fefd3812afc0ddf3184b1451fcff6797df229be53f4dd4 WatchSource:0}: Error finding container ada42bf714f75c66b2fefd3812afc0ddf3184b1451fcff6797df229be53f4dd4: Status 404 returned error can't find the container with id ada42bf714f75c66b2fefd3812afc0ddf3184b1451fcff6797df229be53f4dd4
Sep 09 21:00:14.669401 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.669279    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-428cr"]
Sep 09 21:00:14.675688 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.675660    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr"
Sep 09 21:00:14.680615 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.680593    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8kvtd\""
Sep 09 21:00:14.685311 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.685285    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-428cr"]
Sep 09 21:00:14.734466 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.734407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:14.795310 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.795267    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8vsj\" (UniqueName: \"kubernetes.io/projected/ae4ea1f7-a76e-4079-b050-f3536c247d09-kube-api-access-m8vsj\") pod \"authorino-7bcfd8f9cd-428cr\" (UID: \"ae4ea1f7-a76e-4079-b050-f3536c247d09\") " pod="kuadrant-system/authorino-7bcfd8f9cd-428cr"
Sep 09 21:00:14.829241 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.829211    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w" event={"ID":"e028d0e2-0b0a-46a1-b6da-e64a09b4c767","Type":"ContainerStarted","Data":"ada42bf714f75c66b2fefd3812afc0ddf3184b1451fcff6797df229be53f4dd4"}
Sep 09 21:00:14.835664 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.835623    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:14.896608 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.896567    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8vsj\" (UniqueName: \"kubernetes.io/projected/ae4ea1f7-a76e-4079-b050-f3536c247d09-kube-api-access-m8vsj\") pod \"authorino-7bcfd8f9cd-428cr\" (UID: \"ae4ea1f7-a76e-4079-b050-f3536c247d09\") " pod="kuadrant-system/authorino-7bcfd8f9cd-428cr"
Sep 09 21:00:14.913544 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.913515    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8vsj\" (UniqueName: \"kubernetes.io/projected/ae4ea1f7-a76e-4079-b050-f3536c247d09-kube-api-access-m8vsj\") pod \"authorino-7bcfd8f9cd-428cr\" (UID: \"ae4ea1f7-a76e-4079-b050-f3536c247d09\") " pod="kuadrant-system/authorino-7bcfd8f9cd-428cr"
Sep 09 21:00:14.986571 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:14.986542    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr"
Sep 09 21:00:15.262513 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:15.262482    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-428cr"]
Sep 09 21:00:15.263497 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:00:15.263467    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae4ea1f7_a76e_4079_b050_f3536c247d09.slice/crio-4963f4634ac99243237d133ecf609fa2108fcf007574f12b09cb5c12b390065a WatchSource:0}: Error finding container 4963f4634ac99243237d133ecf609fa2108fcf007574f12b09cb5c12b390065a: Status 404 returned error can't find the container with id 4963f4634ac99243237d133ecf609fa2108fcf007574f12b09cb5c12b390065a
Sep 09 21:00:15.836242 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:15.836203    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr" event={"ID":"ae4ea1f7-a76e-4079-b050-f3536c247d09","Type":"ContainerStarted","Data":"4963f4634ac99243237d133ecf609fa2108fcf007574f12b09cb5c12b390065a"}
Sep 09 21:00:17.591437 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:17.591344    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-428cr"]
Sep 09 21:00:18.860885 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:18.860845    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr" event={"ID":"ae4ea1f7-a76e-4079-b050-f3536c247d09","Type":"ContainerStarted","Data":"b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2"}
Sep 09 21:00:18.861423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:18.860922    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr" podUID="ae4ea1f7-a76e-4079-b050-f3536c247d09" containerName="authorino" containerID="cri-o://b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2" gracePeriod=30
Sep 09 21:00:18.876702 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:18.876653    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr" podStartSLOduration=1.930041987 podStartE2EDuration="4.876636303s" totalImagesPullingTime="2.946594316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:18.875331409 +0000 UTC m=+712.845898127" watchObservedRunningTime="2026-09-09 21:00:18.876636303 +0000 UTC m=+712.847203019"
Sep 09 21:00:19.099284 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.099257    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr"
Sep 09 21:00:19.237778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.237742    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8vsj\" (UniqueName: \"kubernetes.io/projected/ae4ea1f7-a76e-4079-b050-f3536c247d09-kube-api-access-m8vsj\") pod \"ae4ea1f7-a76e-4079-b050-f3536c247d09\" (UID: \"ae4ea1f7-a76e-4079-b050-f3536c247d09\") "
Sep 09 21:00:19.240121 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.240093    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae4ea1f7-a76e-4079-b050-f3536c247d09-kube-api-access-m8vsj" (OuterVolumeSpecName: "kube-api-access-m8vsj") pod "ae4ea1f7-a76e-4079-b050-f3536c247d09" (UID: "ae4ea1f7-a76e-4079-b050-f3536c247d09"). InnerVolumeSpecName "kube-api-access-m8vsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:19.338747 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.338706    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8vsj\" (UniqueName: \"kubernetes.io/projected/ae4ea1f7-a76e-4079-b050-f3536c247d09-kube-api-access-m8vsj\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:00:19.734017 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.733982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:19.868150 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.868110    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w" event={"ID":"e028d0e2-0b0a-46a1-b6da-e64a09b4c767","Type":"ContainerStarted","Data":"c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944"}
Sep 09 21:00:19.868613 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.868178    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:19.869511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.869487    2571 generic.go:356] "Generic (PLEG): container finished" podID="ae4ea1f7-a76e-4079-b050-f3536c247d09" containerID="b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2" exitCode=0
Sep 09 21:00:19.869622 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.869513    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr"
Sep 09 21:00:19.869622 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.869555    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr" event={"ID":"ae4ea1f7-a76e-4079-b050-f3536c247d09","Type":"ContainerDied","Data":"b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2"}
Sep 09 21:00:19.869622 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.869592    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-428cr" event={"ID":"ae4ea1f7-a76e-4079-b050-f3536c247d09","Type":"ContainerDied","Data":"4963f4634ac99243237d133ecf609fa2108fcf007574f12b09cb5c12b390065a"}
Sep 09 21:00:19.869622 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.869606    2571 scope.go:117] "RemoveContainer" containerID="b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2"
Sep 09 21:00:19.879166 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.879148    2571 scope.go:117] "RemoveContainer" containerID="b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2"
Sep 09 21:00:19.879477 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:00:19.879458    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2\": container with ID starting with b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2 not found: ID does not exist" containerID="b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2"
Sep 09 21:00:19.879524 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.879487    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2"} err="failed to get container status \"b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2\": rpc error: code = NotFound desc = could not find container \"b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2\": container with ID starting with b1f360d5ce2f6bca584b83dd168db5b809b74e642a7eb930a756c919b6debdb2 not found: ID does not exist"
Sep 09 21:00:19.887337 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.887293    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w" podStartSLOduration=2.291798726 podStartE2EDuration="6.887277153s" totalImagesPullingTime="4.595478427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:19.884898245 +0000 UTC m=+713.855464963" watchObservedRunningTime="2026-09-09 21:00:19.887277153 +0000 UTC m=+713.857843870"
Sep 09 21:00:19.901613 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.901571    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-428cr"]
Sep 09 21:00:19.904185 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:19.904161    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-428cr"]
Sep 09 21:00:20.566044 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:20.566008    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae4ea1f7-a76e-4079-b050-f3536c247d09" path="/var/lib/kubelet/pods/ae4ea1f7-a76e-4079-b050-f3536c247d09/volumes"
Sep 09 21:00:30.877814 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:30.877780    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:36.398350 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.398308    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4gz6w"]
Sep 09 21:00:36.398788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.398516    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w" podUID="e028d0e2-0b0a-46a1-b6da-e64a09b4c767" containerName="limitador" containerID="cri-o://c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944" gracePeriod=30
Sep 09 21:00:36.950161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.950132    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:36.966181 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.966072    2571 generic.go:356] "Generic (PLEG): container finished" podID="e028d0e2-0b0a-46a1-b6da-e64a09b4c767" containerID="c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944" exitCode=0
Sep 09 21:00:36.966181 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.966151    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w"
Sep 09 21:00:36.966181 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.966168    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w" event={"ID":"e028d0e2-0b0a-46a1-b6da-e64a09b4c767","Type":"ContainerDied","Data":"c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944"}
Sep 09 21:00:36.966441 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.966245    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4gz6w" event={"ID":"e028d0e2-0b0a-46a1-b6da-e64a09b4c767","Type":"ContainerDied","Data":"ada42bf714f75c66b2fefd3812afc0ddf3184b1451fcff6797df229be53f4dd4"}
Sep 09 21:00:36.966441 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.966267    2571 scope.go:117] "RemoveContainer" containerID="c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944"
Sep 09 21:00:36.977116 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.977094    2571 scope.go:117] "RemoveContainer" containerID="c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944"
Sep 09 21:00:36.977468 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:00:36.977449    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944\": container with ID starting with c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944 not found: ID does not exist" containerID="c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944"
Sep 09 21:00:36.977512 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.977479    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944"} err="failed to get container status \"c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944\": rpc error: code = NotFound desc = could not find container \"c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944\": container with ID starting with c68d29bc19919337f4ce8b35e3b48c42faa1a79348b85b0edd22423f411de944 not found: ID does not exist"
Sep 09 21:00:36.998648 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.998612    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-config-file\") pod \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\" (UID: \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\") "
Sep 09 21:00:36.998814 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.998675    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-566v2\" (UniqueName: \"kubernetes.io/projected/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-kube-api-access-566v2\") pod \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\" (UID: \"e028d0e2-0b0a-46a1-b6da-e64a09b4c767\") "
Sep 09 21:00:36.999052 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:36.999026    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-config-file" (OuterVolumeSpecName: "config-file") pod "e028d0e2-0b0a-46a1-b6da-e64a09b4c767" (UID: "e028d0e2-0b0a-46a1-b6da-e64a09b4c767"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:00:37.000912 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:37.000891    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-kube-api-access-566v2" (OuterVolumeSpecName: "kube-api-access-566v2") pod "e028d0e2-0b0a-46a1-b6da-e64a09b4c767" (UID: "e028d0e2-0b0a-46a1-b6da-e64a09b4c767"). InnerVolumeSpecName "kube-api-access-566v2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:37.100364 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:37.100332    2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-config-file\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:00:37.100364 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:37.100360    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-566v2\" (UniqueName: \"kubernetes.io/projected/e028d0e2-0b0a-46a1-b6da-e64a09b4c767-kube-api-access-566v2\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:00:37.289591 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:37.289558    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4gz6w"]
Sep 09 21:00:37.294009 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:37.293981    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4gz6w"]
Sep 09 21:00:38.566482 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:38.566442    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e028d0e2-0b0a-46a1-b6da-e64a09b4c767" path="/var/lib/kubelet/pods/e028d0e2-0b0a-46a1-b6da-e64a09b4c767/volumes"
Sep 09 21:00:39.983556 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:39.983521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:40.080478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.080443    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:40.544536 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.544502    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"]
Sep 09 21:00:40.544935 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.544920    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e028d0e2-0b0a-46a1-b6da-e64a09b4c767" containerName="limitador"
Sep 09 21:00:40.545001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.544936    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e028d0e2-0b0a-46a1-b6da-e64a09b4c767" containerName="limitador"
Sep 09 21:00:40.545001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.544968    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae4ea1f7-a76e-4079-b050-f3536c247d09" containerName="authorino"
Sep 09 21:00:40.545001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.544974    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae4ea1f7-a76e-4079-b050-f3536c247d09" containerName="authorino"
Sep 09 21:00:40.545142 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.545034    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e028d0e2-0b0a-46a1-b6da-e64a09b4c767" containerName="limitador"
Sep 09 21:00:40.545142 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.545043    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae4ea1f7-a76e-4079-b050-f3536c247d09" containerName="authorino"
Sep 09 21:00:40.549586 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.549568    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:40.557117 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.557097    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 21:00:40.557512 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.557493    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 21:00:40.557648 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.557632    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fwvwg\""
Sep 09 21:00:40.557904 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.557888    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 21:00:40.579220 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.579187    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"]
Sep 09 21:00:40.617757 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.617709    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:40.630873 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.630834    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/97276c9b-be1a-4096-8374-58b211f5692d-data\") pod \"postgres-69fb4594cf-hmd4z\" (UID: \"97276c9b-be1a-4096-8374-58b211f5692d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:40.631120 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.631072    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77br6\" (UniqueName: \"kubernetes.io/projected/97276c9b-be1a-4096-8374-58b211f5692d-kube-api-access-77br6\") pod \"postgres-69fb4594cf-hmd4z\" (UID: \"97276c9b-be1a-4096-8374-58b211f5692d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:40.732177 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.732135    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/97276c9b-be1a-4096-8374-58b211f5692d-data\") pod \"postgres-69fb4594cf-hmd4z\" (UID: \"97276c9b-be1a-4096-8374-58b211f5692d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:40.732391 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.732230    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77br6\" (UniqueName: \"kubernetes.io/projected/97276c9b-be1a-4096-8374-58b211f5692d-kube-api-access-77br6\") pod \"postgres-69fb4594cf-hmd4z\" (UID: \"97276c9b-be1a-4096-8374-58b211f5692d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:40.732535 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.732512    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/97276c9b-be1a-4096-8374-58b211f5692d-data\") pod \"postgres-69fb4594cf-hmd4z\" (UID: \"97276c9b-be1a-4096-8374-58b211f5692d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:40.747434 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.747394    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77br6\" (UniqueName: \"kubernetes.io/projected/97276c9b-be1a-4096-8374-58b211f5692d-kube-api-access-77br6\") pod \"postgres-69fb4594cf-hmd4z\" (UID: \"97276c9b-be1a-4096-8374-58b211f5692d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:40.859901 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:40.859808    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:41.140014 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:41.139983    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"]
Sep 09 21:00:41.140429 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:00:41.140142    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97276c9b_be1a_4096_8374_58b211f5692d.slice/crio-72d5951088ac63f43d1a74b83d59c320eab052ac05d8f15c1c76d3772a0f6494 WatchSource:0}: Error finding container 72d5951088ac63f43d1a74b83d59c320eab052ac05d8f15c1c76d3772a0f6494: Status 404 returned error can't find the container with id 72d5951088ac63f43d1a74b83d59c320eab052ac05d8f15c1c76d3772a0f6494
Sep 09 21:00:42.006712 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:42.006667    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z" event={"ID":"97276c9b-be1a-4096-8374-58b211f5692d","Type":"ContainerStarted","Data":"72d5951088ac63f43d1a74b83d59c320eab052ac05d8f15c1c76d3772a0f6494"}
Sep 09 21:00:44.021165 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:44.021129    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z" event={"ID":"97276c9b-be1a-4096-8374-58b211f5692d","Type":"ContainerStarted","Data":"d9288b9353c993e591fab085119eae1945638da9e8ece3efdb136c5b3cf71a87"}
Sep 09 21:00:44.021669 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:44.021199    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:44.037291 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:44.037244    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z" podStartSLOduration=1.970771111 podStartE2EDuration="4.037227075s" totalImagesPullingTime="2.066455964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:44.036218995 +0000 UTC m=+738.006785713" watchObservedRunningTime="2026-09-09 21:00:44.037227075 +0000 UTC m=+738.007793792"
Sep 09 21:00:50.059288 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.059254    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hmd4z"
Sep 09 21:00:50.471524 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.471491    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:50.575788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.575752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:50.587800 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.587764    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-zqc78"]
Sep 09 21:00:50.592852 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.592830    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-zqc78"
Sep 09 21:00:50.595139 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.595116    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8kvtd\""
Sep 09 21:00:50.598555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.598530    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-zqc78"]
Sep 09 21:00:50.722046 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.721957    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-zqc78"]
Sep 09 21:00:50.722321 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:00:50.722298    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jdmb5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-zqc78" podUID="77afff19-0758-4ff5-9a78-cae1c257fff5"
Sep 09 21:00:50.726739 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.726717    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdmb5\" (UniqueName: \"kubernetes.io/projected/77afff19-0758-4ff5-9a78-cae1c257fff5-kube-api-access-jdmb5\") pod \"authorino-557f876f69-zqc78\" (UID: \"77afff19-0758-4ff5-9a78-cae1c257fff5\") " pod="kuadrant-system/authorino-557f876f69-zqc78"
Sep 09 21:00:50.827919 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.827885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdmb5\" (UniqueName: \"kubernetes.io/projected/77afff19-0758-4ff5-9a78-cae1c257fff5-kube-api-access-jdmb5\") pod \"authorino-557f876f69-zqc78\" (UID: \"77afff19-0758-4ff5-9a78-cae1c257fff5\") " pod="kuadrant-system/authorino-557f876f69-zqc78"
Sep 09 21:00:50.843019 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.842976    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdmb5\" (UniqueName: \"kubernetes.io/projected/77afff19-0758-4ff5-9a78-cae1c257fff5-kube-api-access-jdmb5\") pod \"authorino-557f876f69-zqc78\" (UID: \"77afff19-0758-4ff5-9a78-cae1c257fff5\") " pod="kuadrant-system/authorino-557f876f69-zqc78"
Sep 09 21:00:50.893313 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.893283    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75695c59bd-hctkr"]
Sep 09 21:00:50.909598 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.909570    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75695c59bd-hctkr"]
Sep 09 21:00:50.909767 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.909666    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:00:50.911925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:50.911903    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 09 21:00:51.029944 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.029855    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q54q\" (UniqueName: \"kubernetes.io/projected/c1d74778-bc43-4e61-9db0-0b45f95dec58-kube-api-access-9q54q\") pod \"authorino-75695c59bd-hctkr\" (UID: \"c1d74778-bc43-4e61-9db0-0b45f95dec58\") " pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:00:51.029944 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.029936    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1d74778-bc43-4e61-9db0-0b45f95dec58-tls-cert\") pod \"authorino-75695c59bd-hctkr\" (UID: \"c1d74778-bc43-4e61-9db0-0b45f95dec58\") " pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:00:51.062564 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.062536    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-zqc78"
Sep 09 21:00:51.106234 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.106201    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-zqc78"
Sep 09 21:00:51.130552 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.130516    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q54q\" (UniqueName: \"kubernetes.io/projected/c1d74778-bc43-4e61-9db0-0b45f95dec58-kube-api-access-9q54q\") pod \"authorino-75695c59bd-hctkr\" (UID: \"c1d74778-bc43-4e61-9db0-0b45f95dec58\") " pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:00:51.130734 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.130586    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1d74778-bc43-4e61-9db0-0b45f95dec58-tls-cert\") pod \"authorino-75695c59bd-hctkr\" (UID: \"c1d74778-bc43-4e61-9db0-0b45f95dec58\") " pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:00:51.133197 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.133176    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1d74778-bc43-4e61-9db0-0b45f95dec58-tls-cert\") pod \"authorino-75695c59bd-hctkr\" (UID: \"c1d74778-bc43-4e61-9db0-0b45f95dec58\") " pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:00:51.139863 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.139840    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q54q\" (UniqueName: \"kubernetes.io/projected/c1d74778-bc43-4e61-9db0-0b45f95dec58-kube-api-access-9q54q\") pod \"authorino-75695c59bd-hctkr\" (UID: \"c1d74778-bc43-4e61-9db0-0b45f95dec58\") " pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:00:51.219932 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.219894    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:00:51.231274 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.231246    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdmb5\" (UniqueName: \"kubernetes.io/projected/77afff19-0758-4ff5-9a78-cae1c257fff5-kube-api-access-jdmb5\") pod \"77afff19-0758-4ff5-9a78-cae1c257fff5\" (UID: \"77afff19-0758-4ff5-9a78-cae1c257fff5\") "
Sep 09 21:00:51.233514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.233485    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77afff19-0758-4ff5-9a78-cae1c257fff5-kube-api-access-jdmb5" (OuterVolumeSpecName: "kube-api-access-jdmb5") pod "77afff19-0758-4ff5-9a78-cae1c257fff5" (UID: "77afff19-0758-4ff5-9a78-cae1c257fff5"). InnerVolumeSpecName "kube-api-access-jdmb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:00:51.332730 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.332696    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdmb5\" (UniqueName: \"kubernetes.io/projected/77afff19-0758-4ff5-9a78-cae1c257fff5-kube-api-access-jdmb5\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:00:51.517670 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:51.517639    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75695c59bd-hctkr"]
Sep 09 21:00:51.519259 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:00:51.519234    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1d74778_bc43_4e61_9db0_0b45f95dec58.slice/crio-a762eb422c6a26adf2295ca31144b6f9aef2167fb8ea8ab868a65ae6df5034fd WatchSource:0}: Error finding container a762eb422c6a26adf2295ca31144b6f9aef2167fb8ea8ab868a65ae6df5034fd: Status 404 returned error can't find the container with id a762eb422c6a26adf2295ca31144b6f9aef2167fb8ea8ab868a65ae6df5034fd
Sep 09 21:00:52.069896 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:52.069856    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75695c59bd-hctkr" event={"ID":"c1d74778-bc43-4e61-9db0-0b45f95dec58","Type":"ContainerStarted","Data":"a762eb422c6a26adf2295ca31144b6f9aef2167fb8ea8ab868a65ae6df5034fd"}
Sep 09 21:00:52.070354 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:52.069918    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-zqc78"
Sep 09 21:00:52.102254 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:52.102217    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-zqc78"]
Sep 09 21:00:52.106201 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:52.106176    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-zqc78"]
Sep 09 21:00:52.566127 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:52.566061    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77afff19-0758-4ff5-9a78-cae1c257fff5" path="/var/lib/kubelet/pods/77afff19-0758-4ff5-9a78-cae1c257fff5/volumes"
Sep 09 21:00:53.077324 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:53.077278    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75695c59bd-hctkr" event={"ID":"c1d74778-bc43-4e61-9db0-0b45f95dec58","Type":"ContainerStarted","Data":"8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea"}
Sep 09 21:00:53.094322 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:53.094267    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75695c59bd-hctkr" podStartSLOduration=2.4614882209999998 podStartE2EDuration="3.094249958s" totalImagesPullingTime="632.761737ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:00:53.09191619 +0000 UTC m=+747.062482908" watchObservedRunningTime="2026-09-09 21:00:53.094249958 +0000 UTC m=+747.064816674"
Sep 09 21:00:57.889374 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:57.889330    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:57.999500 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:57.999467    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:58.071346 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:58.071314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:58.228680 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:58.228644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:00:58.763475 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:00:58.763427    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:01.972873 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:01.972821    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:02.060756 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:02.060728    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:18.371836 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:18.371797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:31.091402 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:31.091370    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:31.283666 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:31.283628    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:32.638364 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:32.638319    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:32.743604 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:32.743573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:32.980691 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:32.980656    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:33.162997 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.162963    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:33.259263 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.259153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:33.333132 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.333097    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"]
Sep 09 21:01:33.338182 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.338159    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:33.340727 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.340691    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4pppm\""
Sep 09 21:01:33.341011 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.340995    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 21:01:33.356428 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.356399    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"]
Sep 09 21:01:33.373322 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.373286    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:33.438334 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.438295    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03afa2b6-eccc-4aa4-9979-b3b725de7207-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-g7qxr\" (UID: \"03afa2b6-eccc-4aa4-9979-b3b725de7207\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:33.438334 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.438340    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj229\" (UniqueName: \"kubernetes.io/projected/03afa2b6-eccc-4aa4-9979-b3b725de7207-kube-api-access-sj229\") pod \"payload-processing-5b54b7fbb7-g7qxr\" (UID: \"03afa2b6-eccc-4aa4-9979-b3b725de7207\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:33.539866 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.539748    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03afa2b6-eccc-4aa4-9979-b3b725de7207-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-g7qxr\" (UID: \"03afa2b6-eccc-4aa4-9979-b3b725de7207\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:33.539866 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.539810    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj229\" (UniqueName: \"kubernetes.io/projected/03afa2b6-eccc-4aa4-9979-b3b725de7207-kube-api-access-sj229\") pod \"payload-processing-5b54b7fbb7-g7qxr\" (UID: \"03afa2b6-eccc-4aa4-9979-b3b725de7207\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:33.541001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.540966    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03afa2b6-eccc-4aa4-9979-b3b725de7207-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-g7qxr\" (UID: \"03afa2b6-eccc-4aa4-9979-b3b725de7207\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:33.548930 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.548901    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj229\" (UniqueName: \"kubernetes.io/projected/03afa2b6-eccc-4aa4-9979-b3b725de7207-kube-api-access-sj229\") pod \"payload-processing-5b54b7fbb7-g7qxr\" (UID: \"03afa2b6-eccc-4aa4-9979-b3b725de7207\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:33.649230 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.649191    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:33.943710 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:33.943671    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"]
Sep 09 21:01:33.944703 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:01:33.944675    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03afa2b6_eccc_4aa4_9979_b3b725de7207.slice/crio-f310595b99e5c80deba6524af63d686ffa8748efee69f884f6316f625e9de337 WatchSource:0}: Error finding container f310595b99e5c80deba6524af63d686ffa8748efee69f884f6316f625e9de337: Status 404 returned error can't find the container with id f310595b99e5c80deba6524af63d686ffa8748efee69f884f6316f625e9de337
Sep 09 21:01:34.308482 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:34.308445    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr" event={"ID":"03afa2b6-eccc-4aa4-9979-b3b725de7207","Type":"ContainerStarted","Data":"f310595b99e5c80deba6524af63d686ffa8748efee69f884f6316f625e9de337"}
Sep 09 21:01:36.323537 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:36.323498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr" event={"ID":"03afa2b6-eccc-4aa4-9979-b3b725de7207","Type":"ContainerStarted","Data":"6cfbda8e79d220b7bb6bdbfacd064baa0e7d3da2df3fd49ace5a49436d97a216"}
Sep 09 21:01:36.323998 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:36.323626    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:36.342911 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:36.342853    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr" podStartSLOduration=1.467532925 podStartE2EDuration="3.342838034s" totalImagesPullingTime="1.875305109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:01:36.340308153 +0000 UTC m=+790.310874870" watchObservedRunningTime="2026-09-09 21:01:36.342838034 +0000 UTC m=+790.313404783"
Sep 09 21:01:46.033999 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:46.033959    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:47.146738 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:47.146700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:01:47.331166 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:47.331127    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-g7qxr"
Sep 09 21:01:59.707820 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:01:59.707738    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:14.503239 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:14.503205    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:20.212655 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.212612    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"]
Sep 09 21:02:20.218353 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.218328    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.220987 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.220961    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 21:02:20.221583 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.221565    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mnpqm\""
Sep 09 21:02:20.221583 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.221570    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 09 21:02:20.221754 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.221607    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 21:02:20.232854 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.232825    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"]
Sep 09 21:02:20.254610 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.254567    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:20.397856 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.397820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8447f9b-fa9b-43a1-8d30-0e7618775da7-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.397856 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.397856    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbbn2\" (UniqueName: \"kubernetes.io/projected/f8447f9b-fa9b-43a1-8d30-0e7618775da7-kube-api-access-gbbn2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.398108 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.397895    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.398108 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.397972    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.398108 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.397997    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.398108 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.398012    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.499517 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.499425    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8447f9b-fa9b-43a1-8d30-0e7618775da7-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.499517 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.499465    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbbn2\" (UniqueName: \"kubernetes.io/projected/f8447f9b-fa9b-43a1-8d30-0e7618775da7-kube-api-access-gbbn2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.499517 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.499497    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.499783 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.499543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.499783 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.499563    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.499783 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.499575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.500105 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.500056    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.500202 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.500071    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.500202 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.500148    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.502141 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.502122    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f8447f9b-fa9b-43a1-8d30-0e7618775da7-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.502233 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.502223    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8447f9b-fa9b-43a1-8d30-0e7618775da7-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.509018 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.508991    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbbn2\" (UniqueName: \"kubernetes.io/projected/f8447f9b-fa9b-43a1-8d30-0e7618775da7-kube-api-access-gbbn2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx\" (UID: \"f8447f9b-fa9b-43a1-8d30-0e7618775da7\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.530735 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.530699    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:20.807715 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:20.807686    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"]
Sep 09 21:02:20.809141 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:02:20.809114    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8447f9b_fa9b_43a1_8d30_0e7618775da7.slice/crio-a18c64cb4ddf57966d7465741fd2227f8456b0a249963314a3698a9a46ebe453 WatchSource:0}: Error finding container a18c64cb4ddf57966d7465741fd2227f8456b0a249963314a3698a9a46ebe453: Status 404 returned error can't find the container with id a18c64cb4ddf57966d7465741fd2227f8456b0a249963314a3698a9a46ebe453
Sep 09 21:02:21.590642 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:21.590546    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx" event={"ID":"f8447f9b-fa9b-43a1-8d30-0e7618775da7","Type":"ContainerStarted","Data":"a18c64cb4ddf57966d7465741fd2227f8456b0a249963314a3698a9a46ebe453"}
Sep 09 21:02:22.598274 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:22.598238    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx" event={"ID":"f8447f9b-fa9b-43a1-8d30-0e7618775da7","Type":"ContainerStarted","Data":"1918f1b87c60775eb91ced45fec1357fed619abedac359c8f75166c597583445"}
Sep 09 21:02:22.616231 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:22.616177    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx" podStartSLOduration=1.5186443440000001 podStartE2EDuration="2.616160254s" totalImagesPullingTime="1.09751591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:22.614861991 +0000 UTC m=+836.585428707" watchObservedRunningTime="2026-09-09 21:02:22.616160254 +0000 UTC m=+836.586726970"
Sep 09 21:02:26.121611 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.121577    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"]
Sep 09 21:02:26.125497 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.125479    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.127972 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.127954    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 09 21:02:26.136931 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.136907    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"]
Sep 09 21:02:26.140970 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.140944    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:26.160119 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.160061    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3e6df7d-f587-432e-b8ff-863913e5674e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.160326 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.160161    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.160326 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.160204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.160326 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.160234    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.160464 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.160323    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qdpv\" (UniqueName: \"kubernetes.io/projected/b3e6df7d-f587-432e-b8ff-863913e5674e-kube-api-access-5qdpv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.160464 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.160358    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261275 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261239    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261531 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261305    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3e6df7d-f587-432e-b8ff-863913e5674e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261531 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261327    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261531 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261352    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261531 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261367    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261531 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261414    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qdpv\" (UniqueName: \"kubernetes.io/projected/b3e6df7d-f587-432e-b8ff-863913e5674e-kube-api-access-5qdpv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261791 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261764    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261854 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261815    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.261854 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.261841    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.263787 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.263767    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3e6df7d-f587-432e-b8ff-863913e5674e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.263950 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.263933    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3e6df7d-f587-432e-b8ff-863913e5674e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.270455 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.270430    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qdpv\" (UniqueName: \"kubernetes.io/projected/b3e6df7d-f587-432e-b8ff-863913e5674e-kube-api-access-5qdpv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vssg4\" (UID: \"b3e6df7d-f587-432e-b8ff-863913e5674e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.436287 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.436183    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:26.712248 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:26.712219    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"]
Sep 09 21:02:26.714044 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:02:26.714017    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e6df7d_f587_432e_b8ff_863913e5674e.slice/crio-7c0c30961ae246a3ed51b70c7e283b805b6c2c53e065470be6653e0d6c7edc30 WatchSource:0}: Error finding container 7c0c30961ae246a3ed51b70c7e283b805b6c2c53e065470be6653e0d6c7edc30: Status 404 returned error can't find the container with id 7c0c30961ae246a3ed51b70c7e283b805b6c2c53e065470be6653e0d6c7edc30
Sep 09 21:02:27.630492 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:27.630447    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4" event={"ID":"b3e6df7d-f587-432e-b8ff-863913e5674e","Type":"ContainerStarted","Data":"0512060f178b880e7be4ee3e93f64e32a1480bee050e4d49193bd88fac118c53"}
Sep 09 21:02:27.631004 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:27.630520    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4" event={"ID":"b3e6df7d-f587-432e-b8ff-863913e5674e","Type":"ContainerStarted","Data":"7c0c30961ae246a3ed51b70c7e283b805b6c2c53e065470be6653e0d6c7edc30"}
Sep 09 21:02:27.649555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:27.649487    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4" podStartSLOduration=1.473779855 podStartE2EDuration="1.649463876s" totalImagesPullingTime="175.684021ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:02:27.648650726 +0000 UTC m=+841.619217442" watchObservedRunningTime="2026-09-09 21:02:27.649463876 +0000 UTC m=+841.620030595"
Sep 09 21:02:30.530988 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:30.530952    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:30.531419 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:30.531051    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:30.543679 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:30.543650    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:30.659253 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:30.659225    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx"
Sep 09 21:02:36.436748 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:36.436711    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:36.437255 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:36.436780    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:36.449831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:36.449809    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:36.586368 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:36.586326    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:36.695489 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:36.695459    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vssg4"
Sep 09 21:02:42.835926 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:42.835891    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:48.861722 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:48.861688    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:02:59.248068 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:02:59.248028    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:03:22.154215 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:03:22.154132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:12.786922 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:12.786863    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75695c59bd-hctkr"]
Sep 09 21:04:12.787506 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:12.787144    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75695c59bd-hctkr" podUID="c1d74778-bc43-4e61-9db0-0b45f95dec58" containerName="authorino" containerID="cri-o://8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea" gracePeriod=30
Sep 09 21:04:13.046348 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.046282    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:04:13.167940 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.167909    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q54q\" (UniqueName: \"kubernetes.io/projected/c1d74778-bc43-4e61-9db0-0b45f95dec58-kube-api-access-9q54q\") pod \"c1d74778-bc43-4e61-9db0-0b45f95dec58\" (UID: \"c1d74778-bc43-4e61-9db0-0b45f95dec58\") "
Sep 09 21:04:13.168171 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.168061    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1d74778-bc43-4e61-9db0-0b45f95dec58-tls-cert\") pod \"c1d74778-bc43-4e61-9db0-0b45f95dec58\" (UID: \"c1d74778-bc43-4e61-9db0-0b45f95dec58\") "
Sep 09 21:04:13.170341 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.170310    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d74778-bc43-4e61-9db0-0b45f95dec58-kube-api-access-9q54q" (OuterVolumeSpecName: "kube-api-access-9q54q") pod "c1d74778-bc43-4e61-9db0-0b45f95dec58" (UID: "c1d74778-bc43-4e61-9db0-0b45f95dec58"). InnerVolumeSpecName "kube-api-access-9q54q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:04:13.179648 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.179622    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1d74778-bc43-4e61-9db0-0b45f95dec58-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c1d74778-bc43-4e61-9db0-0b45f95dec58" (UID: "c1d74778-bc43-4e61-9db0-0b45f95dec58"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:04:13.257708 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.257672    2571 generic.go:356] "Generic (PLEG): container finished" podID="c1d74778-bc43-4e61-9db0-0b45f95dec58" containerID="8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea" exitCode=0
Sep 09 21:04:13.257891 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.257717    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75695c59bd-hctkr"
Sep 09 21:04:13.257891 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.257770    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75695c59bd-hctkr" event={"ID":"c1d74778-bc43-4e61-9db0-0b45f95dec58","Type":"ContainerDied","Data":"8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea"}
Sep 09 21:04:13.257891 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.257810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75695c59bd-hctkr" event={"ID":"c1d74778-bc43-4e61-9db0-0b45f95dec58","Type":"ContainerDied","Data":"a762eb422c6a26adf2295ca31144b6f9aef2167fb8ea8ab868a65ae6df5034fd"}
Sep 09 21:04:13.257891 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.257830    2571 scope.go:117] "RemoveContainer" containerID="8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea"
Sep 09 21:04:13.268551 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.268522    2571 scope.go:117] "RemoveContainer" containerID="8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea"
Sep 09 21:04:13.268964 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:13.268930    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea\": container with ID starting with 8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea not found: ID does not exist" containerID="8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea"
Sep 09 21:04:13.269062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.268990    2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c1d74778-bc43-4e61-9db0-0b45f95dec58-tls-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:04:13.269062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.268978    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea"} err="failed to get container status \"8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea\": rpc error: code = NotFound desc = could not find container \"8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea\": container with ID starting with 8e0e3d502fed9f4854d5e07411c45a192d664803f4ff4ee3a836817133f4eeea not found: ID does not exist"
Sep 09 21:04:13.269062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.269011    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q54q\" (UniqueName: \"kubernetes.io/projected/c1d74778-bc43-4e61-9db0-0b45f95dec58-kube-api-access-9q54q\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:04:13.290274 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.290231    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75695c59bd-hctkr"]
Sep 09 21:04:13.292371 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.292336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:13.295903 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.295881    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75695c59bd-hctkr"]
Sep 09 21:04:13.539300 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:13.539253    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:14.431514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:14.431467    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:14.527667 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:14.527624    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:14.573346 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:14.573310    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1d74778-bc43-4e61-9db0-0b45f95dec58" path="/var/lib/kubelet/pods/c1d74778-bc43-4e61-9db0-0b45f95dec58/volumes"
Sep 09 21:04:29.597938 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:29.597900    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:29.696049 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:29.696005    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:29.991443 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:29.991410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:30.151214 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:30.151176    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:31.060533 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.060477    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:31.171051 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.171010    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:31.521693 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.521659    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:31.603054 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.603018    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-95fb68"]
Sep 09 21:04:31.603469 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.603457    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1d74778-bc43-4e61-9db0-0b45f95dec58" containerName="authorino"
Sep 09 21:04:31.603514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.603471    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1d74778-bc43-4e61-9db0-0b45f95dec58" containerName="authorino"
Sep 09 21:04:31.603554 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.603540    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1d74778-bc43-4e61-9db0-0b45f95dec58" containerName="authorino"
Sep 09 21:04:31.607903 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.607868    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-95fb68"
Sep 09 21:04:31.610018 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.609990    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-95fb68"]
Sep 09 21:04:31.610293 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.610277    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cmlrc\""
Sep 09 21:04:31.750069 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.750039    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xhx2\" (UniqueName: \"kubernetes.io/projected/cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b-kube-api-access-6xhx2\") pod \"test-curl-7613-95fb68\" (UID: \"cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b\") " pod="opendatahub/test-curl-7613-95fb68"
Sep 09 21:04:31.850651 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.850565    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xhx2\" (UniqueName: \"kubernetes.io/projected/cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b-kube-api-access-6xhx2\") pod \"test-curl-7613-95fb68\" (UID: \"cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b\") " pod="opendatahub/test-curl-7613-95fb68"
Sep 09 21:04:31.864510 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.864475    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xhx2\" (UniqueName: \"kubernetes.io/projected/cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b-kube-api-access-6xhx2\") pod \"test-curl-7613-95fb68\" (UID: \"cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b\") " pod="opendatahub/test-curl-7613-95fb68"
Sep 09 21:04:31.918269 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:31.918228    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-95fb68"
Sep 09 21:04:32.192421 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:32.192391    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-95fb68"]
Sep 09 21:04:32.194113 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:04:32.194056    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfcd8b0c_4d68_4ca5_9864_bb17ac4ae30b.slice/crio-c6be6527426ad2a59957bc0e07f8a1a05222e1084152ce0346a5d011349f10da WatchSource:0}: Error finding container c6be6527426ad2a59957bc0e07f8a1a05222e1084152ce0346a5d011349f10da: Status 404 returned error can't find the container with id c6be6527426ad2a59957bc0e07f8a1a05222e1084152ce0346a5d011349f10da
Sep 09 21:04:32.195921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:32.195904    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:04:32.371994 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:32.371962    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-95fb68" event={"ID":"cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b","Type":"ContainerStarted","Data":"c6be6527426ad2a59957bc0e07f8a1a05222e1084152ce0346a5d011349f10da"}
Sep 09 21:04:32.770037 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:32.769997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:32.874592 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:32.874547    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:33.912418 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.912362    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"]
Sep 09 21:04:33.922325 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.922297    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:33.926674 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.926323    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-gateway-identity\""
Sep 09 21:04:33.929466 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.927036    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gj4rp\""
Sep 09 21:04:33.929466 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.927330    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 21:04:33.929466 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.927516    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9a54ea-serving-cert\""
Sep 09 21:04:33.933181 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.933055    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"]
Sep 09 21:04:33.970568 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.970530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:33.970568 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.970580    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30dca177-8dab-4d50-9d08-91b3e7da1018-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:33.970832 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.970688    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-metrics-certs\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:33.970832 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.970721    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jltlf\" (UniqueName: \"kubernetes.io/projected/30dca177-8dab-4d50-9d08-91b3e7da1018-kube-api-access-jltlf\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:33.974007 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:33.973957    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:34.068938 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.068900    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:34.071937 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.071896    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.072071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.071939    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30dca177-8dab-4d50-9d08-91b3e7da1018-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.072071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.072024    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-metrics-certs\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.072071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.072051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jltlf\" (UniqueName: \"kubernetes.io/projected/30dca177-8dab-4d50-9d08-91b3e7da1018-kube-api-access-jltlf\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.072282 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:34.072129    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-serving-cert: secret "maas-api-e2e-inf-a-9a54ea-serving-cert" not found
Sep 09 21:04:34.072282 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:34.072234    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls podName:30dca177-8dab-4d50-9d08-91b3e7da1018 nodeName:}" failed. No retries permitted until 2026-09-09 21:04:34.572210306 +0000 UTC m=+968.542777004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls") pod "maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp" (UID: "30dca177-8dab-4d50-9d08-91b3e7da1018") : secret "maas-api-e2e-inf-a-9a54ea-serving-cert" not found
Sep 09 21:04:34.073095 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.073048    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30dca177-8dab-4d50-9d08-91b3e7da1018-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.075484 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.075460    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-metrics-certs\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.097156 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.097123    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jltlf\" (UniqueName: \"kubernetes.io/projected/30dca177-8dab-4d50-9d08-91b3e7da1018-kube-api-access-jltlf\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.169745 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.169662    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:34.577934 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.577905    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.580865 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.580831    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls\") pod \"maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:34.633281 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.633239    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:34.728647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.728606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:34.842696 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:34.842606    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:35.121705 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:35.121676    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"]
Sep 09 21:04:35.123490 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:04:35.123459    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30dca177_8dab_4d50_9d08_91b3e7da1018.slice/crio-39261b79be0e04f166aa5e4f0ffbdf7914117884183dffa02883f70edb52ad08 WatchSource:0}: Error finding container 39261b79be0e04f166aa5e4f0ffbdf7914117884183dffa02883f70edb52ad08: Status 404 returned error can't find the container with id 39261b79be0e04f166aa5e4f0ffbdf7914117884183dffa02883f70edb52ad08
Sep 09 21:04:35.398565 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:35.398471    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-95fb68" event={"ID":"cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b","Type":"ContainerStarted","Data":"08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219"}
Sep 09 21:04:35.399743 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:35.399718    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp" event={"ID":"30dca177-8dab-4d50-9d08-91b3e7da1018","Type":"ContainerStarted","Data":"39261b79be0e04f166aa5e4f0ffbdf7914117884183dffa02883f70edb52ad08"}
Sep 09 21:04:35.414188 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:35.414142    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7613-95fb68" podStartSLOduration=2.100991191 podStartE2EDuration="4.41412803s" totalImagesPullingTime="2.313136839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:35.412199709 +0000 UTC m=+969.382766427" watchObservedRunningTime="2026-09-09 21:04:35.41412803 +0000 UTC m=+969.384694747"
Sep 09 21:04:36.076790 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.076725    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-95fb68"]
Sep 09 21:04:36.080943 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.080884    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-95fb68"]
Sep 09 21:04:36.196804 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.196767    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7613-80d7a1"]
Sep 09 21:04:36.197431 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.197383    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b" containerName="test-curl-7613-95fb68"
Sep 09 21:04:36.197431 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.197402    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b" containerName="test-curl-7613-95fb68"
Sep 09 21:04:36.197551 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.197509    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b" containerName="test-curl-7613-95fb68"
Sep 09 21:04:36.200909 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.200887    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:36.204524 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.204490    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-80d7a1"]
Sep 09 21:04:36.295661 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.295608    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4lgm\" (UniqueName: \"kubernetes.io/projected/156647cc-c50d-47c0-99c9-31fbc946e62a-kube-api-access-f4lgm\") pod \"test-curl-7613-80d7a1\" (UID: \"156647cc-c50d-47c0-99c9-31fbc946e62a\") " pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:36.396960 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.396924    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4lgm\" (UniqueName: \"kubernetes.io/projected/156647cc-c50d-47c0-99c9-31fbc946e62a-kube-api-access-f4lgm\") pod \"test-curl-7613-80d7a1\" (UID: \"156647cc-c50d-47c0-99c9-31fbc946e62a\") " pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:36.405548 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.405521    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4lgm\" (UniqueName: \"kubernetes.io/projected/156647cc-c50d-47c0-99c9-31fbc946e62a-kube-api-access-f4lgm\") pod \"test-curl-7613-80d7a1\" (UID: \"156647cc-c50d-47c0-99c9-31fbc946e62a\") " pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:36.407746 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:36.407720    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7613-95fb68" container="test-curl-7613-95fb68"
Sep 09 21:04:36.516550 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:36.516478    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:37.416329 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:37.416282    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7613-95fb68" podUID="cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b" containerName="test-curl-7613-95fb68" containerID="cri-o://08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219" gracePeriod=2
Sep 09 21:04:37.483630 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:37.483590    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:37.580359 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:37.580327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:37.680013 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:37.679930    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7613-80d7a1"]
Sep 09 21:04:37.682807 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:04:37.682779    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod156647cc_c50d_47c0_99c9_31fbc946e62a.slice/crio-0b3062d87d6d78c20dc7640b63ee2bf940126ad1c01cdea8fe3cf7615af9653a WatchSource:0}: Error finding container 0b3062d87d6d78c20dc7640b63ee2bf940126ad1c01cdea8fe3cf7615af9653a: Status 404 returned error can't find the container with id 0b3062d87d6d78c20dc7640b63ee2bf940126ad1c01cdea8fe3cf7615af9653a
Sep 09 21:04:37.699901 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:37.699861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:37.987323 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:37.987272    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:38.423574 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:38.423465    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp" event={"ID":"30dca177-8dab-4d50-9d08-91b3e7da1018","Type":"ContainerStarted","Data":"0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933"}
Sep 09 21:04:38.423574 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:38.423571    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:38.425062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:38.425038    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-80d7a1" event={"ID":"156647cc-c50d-47c0-99c9-31fbc946e62a","Type":"ContainerStarted","Data":"1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45"}
Sep 09 21:04:38.425174 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:38.425071    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7613-80d7a1" event={"ID":"156647cc-c50d-47c0-99c9-31fbc946e62a","Type":"ContainerStarted","Data":"0b3062d87d6d78c20dc7640b63ee2bf940126ad1c01cdea8fe3cf7615af9653a"}
Sep 09 21:04:38.443088 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:38.443029    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp" podStartSLOduration=3.101790669 podStartE2EDuration="5.443009921s" totalImagesPullingTime="2.341219252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:38.439957104 +0000 UTC m=+972.410523821" watchObservedRunningTime="2026-09-09 21:04:38.443009921 +0000 UTC m=+972.413576638"
Sep 09 21:04:38.454906 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:38.454836    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7613-80d7a1" podStartSLOduration=2.454812054 podStartE2EDuration="2.454812054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:38.454521329 +0000 UTC m=+972.425088046" watchObservedRunningTime="2026-09-09 21:04:38.454812054 +0000 UTC m=+972.425378772"
Sep 09 21:04:38.643793 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:38.643766    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:39.067423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.067390    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7613-80d7a1"]
Sep 09 21:04:39.070997 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.070968    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7613-80d7a1"]
Sep 09 21:04:39.432268 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:39.432239    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7613-80d7a1" container="test-curl-7613-80d7a1"
Sep 09 21:04:39.581071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.581044    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:39.587365 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.587344    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-95fb68"
Sep 09 21:04:39.628667 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.628635    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xhx2\" (UniqueName: \"kubernetes.io/projected/cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b-kube-api-access-6xhx2\") pod \"cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b\" (UID: \"cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b\") "
Sep 09 21:04:39.631022 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.630997    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b-kube-api-access-6xhx2" (OuterVolumeSpecName: "kube-api-access-6xhx2") pod "cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b" (UID: "cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b"). InnerVolumeSpecName "kube-api-access-6xhx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:04:39.730110 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.730030    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xhx2\" (UniqueName: \"kubernetes.io/projected/cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b-kube-api-access-6xhx2\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:04:39.840897 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.840852    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"]
Sep 09 21:04:39.841365 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.841343    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" containerName="test-curl-7613-80d7a1"
Sep 09 21:04:39.841486 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.841367    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" containerName="test-curl-7613-80d7a1"
Sep 09 21:04:39.841486 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.841481    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" containerName="test-curl-7613-80d7a1"
Sep 09 21:04:39.844763 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.844745    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.847069 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.847046    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 21:04:39.847193 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.847046    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-80db9b-openshift-default-dockercfg-svq8l\""
Sep 09 21:04:39.862055 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.862023    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"]
Sep 09 21:04:39.888422 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.888390    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:39.932177 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932143    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-credential-socket\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.932343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932189    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-certs\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.932343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932256    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4ggg\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-kube-api-access-q4ggg\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.932343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932327    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-token\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.932478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932362    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-podinfo\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.932478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932419    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-data\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.932478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932445    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-socket\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.932569 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932479    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27f8f5e-6c53-400d-9890-a494ec75545a-istiod-ca-cert\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:39.932569 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:39.932495    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-envoy\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033284 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033197    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-podinfo\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033284 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033248    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-data\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033284 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-socket\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033558 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033390    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27f8f5e-6c53-400d-9890-a494ec75545a-istiod-ca-cert\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033558 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033422    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-envoy\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033558 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033498    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-credential-socket\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033558 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033539    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-certs\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033734 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033576    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4ggg\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-kube-api-access-q4ggg\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033734 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033621    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-token\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033734 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033668    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-socket\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.033890 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033843    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-data\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.034022 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.033983    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-credential-socket\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.034022 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.034008    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-certs\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.035071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.035022    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27f8f5e-6c53-400d-9890-a494ec75545a-istiod-ca-cert\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.036232 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.036207    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-envoy\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.036429 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.036407    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-podinfo\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.043901 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.043879    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-token\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.044041 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.044011    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4ggg\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-kube-api-access-q4ggg\") pod \"e2e-autores-80db9b-openshift-default-86f7654d7b-srpww\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") " pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.092217 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.092182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:40.157961 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.157932    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:40.440013 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.439930    2571 generic.go:356] "Generic (PLEG): container finished" podID="cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b" containerID="08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219" exitCode=137
Sep 09 21:04:40.440013 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.439975    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-95fb68"
Sep 09 21:04:40.440492 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.440033    2571 scope.go:117] "RemoveContainer" containerID="08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219"
Sep 09 21:04:40.440492 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.440274    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7613-80d7a1" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" containerName="test-curl-7613-80d7a1" containerID="cri-o://1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45" gracePeriod=2
Sep 09 21:04:40.442758 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:40.442722    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-80d7a1\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:40.450064 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.450043    2571 scope.go:117] "RemoveContainer" containerID="08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219"
Sep 09 21:04:40.450408 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:40.450382    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219\": container with ID starting with 08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219 not found: ID does not exist" containerID="08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219"
Sep 09 21:04:40.450468 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.450422    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219"} err="failed to get container status \"08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219\": rpc error: code = NotFound desc = could not find container \"08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219\": container with ID starting with 08a5bae04d4f4f22adeb8a7e49d33538bf5bc2c6f6260ad7120dc38d2cb68219 not found: ID does not exist"
Sep 09 21:04:40.460247 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:40.460213    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-80d7a1\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:40.462544 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:40.462522    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-95fb68\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b" pod="opendatahub/test-curl-7613-95fb68"
Sep 09 21:04:40.470413 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:04:40.470374    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf27f8f5e_6c53_400d_9890_a494ec75545a.slice/crio-eb4a3146ce6d9878074314445d793d95c19d7856cb51a97023eb7650b4bb0a68 WatchSource:0}: Error finding container eb4a3146ce6d9878074314445d793d95c19d7856cb51a97023eb7650b4bb0a68: Status 404 returned error can't find the container with id eb4a3146ce6d9878074314445d793d95c19d7856cb51a97023eb7650b4bb0a68
Sep 09 21:04:40.470560 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.470539    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"]
Sep 09 21:04:40.569755 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.569712    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b" path="/var/lib/kubelet/pods/cfcd8b0c-4d68-4ca5-9864-bb17ac4ae30b/volumes"
Sep 09 21:04:40.765300 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:40.765266    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:41.450588 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:41.450538    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" event={"ID":"f27f8f5e-6c53-400d-9890-a494ec75545a","Type":"ContainerStarted","Data":"eb4a3146ce6d9878074314445d793d95c19d7856cb51a97023eb7650b4bb0a68"}
Sep 09 21:04:43.212992 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.212951    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 21:04:43.213338 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.213030    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 21:04:43.213338 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.213100    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 21:04:43.299460 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.299423    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:43.369358 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.369323    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4lgm\" (UniqueName: \"kubernetes.io/projected/156647cc-c50d-47c0-99c9-31fbc946e62a-kube-api-access-f4lgm\") pod \"156647cc-c50d-47c0-99c9-31fbc946e62a\" (UID: \"156647cc-c50d-47c0-99c9-31fbc946e62a\") "
Sep 09 21:04:43.371628 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.371587    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/156647cc-c50d-47c0-99c9-31fbc946e62a-kube-api-access-f4lgm" (OuterVolumeSpecName: "kube-api-access-f4lgm") pod "156647cc-c50d-47c0-99c9-31fbc946e62a" (UID: "156647cc-c50d-47c0-99c9-31fbc946e62a"). InnerVolumeSpecName "kube-api-access-f4lgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:04:43.468572 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.468534    2571 generic.go:356] "Generic (PLEG): container finished" podID="156647cc-c50d-47c0-99c9-31fbc946e62a" containerID="1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45" exitCode=137
Sep 09 21:04:43.468572 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.468573    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:43.468871 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.468633    2571 scope.go:117] "RemoveContainer" containerID="1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45"
Sep 09 21:04:43.470204 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.470044    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4lgm\" (UniqueName: \"kubernetes.io/projected/156647cc-c50d-47c0-99c9-31fbc946e62a-kube-api-access-f4lgm\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:04:43.470663 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.470632    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" event={"ID":"f27f8f5e-6c53-400d-9890-a494ec75545a","Type":"ContainerStarted","Data":"30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2"}
Sep 09 21:04:43.481544 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.481522    2571 scope.go:117] "RemoveContainer" containerID="1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45"
Sep 09 21:04:43.481910 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:43.481884    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45\": container with ID starting with 1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45 not found: ID does not exist" containerID="1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45"
Sep 09 21:04:43.482026 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.481914    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45"} err="failed to get container status \"1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45\": rpc error: code = NotFound desc = could not find container \"1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45\": container with ID starting with 1da543183195b64b5cf02b092cb2ff60be80070030bd456978daf254f100dd45 not found: ID does not exist"
Sep 09 21:04:43.493982 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.493925    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" podStartSLOduration=1.754226777 podStartE2EDuration="4.493905523s" totalImagesPullingTime="2.739678746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:43.492955969 +0000 UTC m=+977.463522688" watchObservedRunningTime="2026-09-09 21:04:43.493905523 +0000 UTC m=+977.464472240"
Sep 09 21:04:43.495147 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:43.495113    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-80d7a1\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:43.689133 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.689071    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:43.798741 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:43.798658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:44.158873 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:44.158773    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:44.163865 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:44.163836    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:44.187864 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:44.187819    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-80d7a1\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:44.437507 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:44.437425    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:04:44.439833 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:44.439806    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-80d7a1\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:44.478526 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:44.478498    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:44.479609 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:44.479587    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:04:44.503379 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:04:44.503339    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7613-80d7a1\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" pod="opendatahub/test-curl-7613-80d7a1"
Sep 09 21:04:44.566177 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:44.566146    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="156647cc-c50d-47c0-99c9-31fbc946e62a" path="/var/lib/kubelet/pods/156647cc-c50d-47c0-99c9-31fbc946e62a/volumes"
Sep 09 21:04:44.919888 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:44.919849    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:45.043255 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:45.043226    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:45.423592 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:45.423559    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:45.514930 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:45.514897    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:45.870674 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:45.870635    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:46.170408 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:46.170316    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:47.208761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:47.208724    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:47.313333 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:47.313266    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:47.774147 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:47.774103    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:47.913117 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:47.913053    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:48.006352 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.006314    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"]
Sep 09 21:04:48.011212 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.011194    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:48.014822 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.014799    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-80db9b-dockercfg-nvwqx\""
Sep 09 21:04:48.015001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.014984    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-80db9b\""
Sep 09 21:04:48.028194 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.028119    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"]
Sep 09 21:04:48.082935 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.082901    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"]
Sep 09 21:04:48.086634 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.086616    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:48.108112 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.108059    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"]
Sep 09 21:04:48.120925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.120896    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl85q\" (UniqueName: \"kubernetes.io/projected/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-kube-api-access-xl85q\") pod \"payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m\" (UID: \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:48.121059 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.120993    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m\" (UID: \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:48.222557 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.222515    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-plugins-config-volume\") pod \"payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l\" (UID: \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\") " pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:48.222557 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.222566    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl85q\" (UniqueName: \"kubernetes.io/projected/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-kube-api-access-xl85q\") pod \"payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m\" (UID: \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:48.223073 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.222601    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff5sp\" (UniqueName: \"kubernetes.io/projected/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-kube-api-access-ff5sp\") pod \"payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l\" (UID: \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\") " pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:48.223073 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.222645    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m\" (UID: \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:48.223464 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.223444    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m\" (UID: \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:48.234594 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.234562    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl85q\" (UniqueName: \"kubernetes.io/projected/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-kube-api-access-xl85q\") pod \"payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m\" (UID: \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:48.321367 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.321216    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:48.323691 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.323659    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-plugins-config-volume\") pod \"payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l\" (UID: \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\") " pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:48.323801 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.323722    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff5sp\" (UniqueName: \"kubernetes.io/projected/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-kube-api-access-ff5sp\") pod \"payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l\" (UID: \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\") " pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:48.324718 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.324441    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-plugins-config-volume\") pod \"payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l\" (UID: \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\") " pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:48.332961 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.332933    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff5sp\" (UniqueName: \"kubernetes.io/projected/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-kube-api-access-ff5sp\") pod \"payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l\" (UID: \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\") " pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:48.397423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.397393    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:48.488375 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.488000    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:48.596716 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.596633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:48.610621 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:04:48.610590    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a02e01f_0c6a_4c89_b3ae_8d6de3969ee4.slice/crio-4c032dcbb0fd7d00e269f7dbb48bc27f6f547a0abd59aa4257f5a2e68cbf4de9 WatchSource:0}: Error finding container 4c032dcbb0fd7d00e269f7dbb48bc27f6f547a0abd59aa4257f5a2e68cbf4de9: Status 404 returned error can't find the container with id 4c032dcbb0fd7d00e269f7dbb48bc27f6f547a0abd59aa4257f5a2e68cbf4de9
Sep 09 21:04:48.610886 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.610868    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"]
Sep 09 21:04:48.696503 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.696477    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"]
Sep 09 21:04:48.697502 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:04:48.697480    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61511b0c_3fc1_4a3b_a1e5_fcb8bdd4b6de.slice/crio-1d59ef88a5028bf94db39f55ef989b80b7f71229aeb6c222e2ed100ae0d57282 WatchSource:0}: Error finding container 1d59ef88a5028bf94db39f55ef989b80b7f71229aeb6c222e2ed100ae0d57282: Status 404 returned error can't find the container with id 1d59ef88a5028bf94db39f55ef989b80b7f71229aeb6c222e2ed100ae0d57282
Sep 09 21:04:48.710013 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.709986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:48.829514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.829457    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:48.978902 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:48.978857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:49.499883 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.499845    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:49.518638 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.518600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m" event={"ID":"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4","Type":"ContainerStarted","Data":"0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442"}
Sep 09 21:04:49.518806 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.518703    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m" event={"ID":"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4","Type":"ContainerStarted","Data":"4c032dcbb0fd7d00e269f7dbb48bc27f6f547a0abd59aa4257f5a2e68cbf4de9"}
Sep 09 21:04:49.518806 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.518734    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:04:49.520407 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.520378    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l" event={"ID":"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de","Type":"ContainerStarted","Data":"76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631"}
Sep 09 21:04:49.520543 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.520415    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l" event={"ID":"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de","Type":"ContainerStarted","Data":"1d59ef88a5028bf94db39f55ef989b80b7f71229aeb6c222e2ed100ae0d57282"}
Sep 09 21:04:49.520543 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.520521    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:04:49.538777 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.538723    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m" podStartSLOduration=2.538706708 podStartE2EDuration="2.538706708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:49.537001753 +0000 UTC m=+983.507568465" watchObservedRunningTime="2026-09-09 21:04:49.538706708 +0000 UTC m=+983.509273425"
Sep 09 21:04:49.557864 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:49.557792    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l" podStartSLOduration=1.557775029 podStartE2EDuration="1.557775029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:04:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:04:49.555694664 +0000 UTC m=+983.526261382" watchObservedRunningTime="2026-09-09 21:04:49.557775029 +0000 UTC m=+983.528341757"
Sep 09 21:04:51.556960 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:51.556917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:53.123988 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:53.123957    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:56.378029 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:56.377990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:57.472278 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:57.472233    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:57.586128 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:57.586096    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:57.687210 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:57.687150    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:57.847520 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:57.847415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:58.099005 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:58.098922    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:58.387201 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:58.387106    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:04:58.522828 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:04:58.522788    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:00.199787 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:00.199750    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:00.519397 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:00.519293    2571 patch_prober.go:28] interesting pod/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 21:05:00.519397 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:00.519364    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 21:05:00.528258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:00.528229    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:05:00.528444 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:00.528318    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:05:01.406186 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:01.406149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:01.591294 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:01.591261    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:02.612909 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:02.612802    2571 generic.go:356] "Generic (PLEG): container finished" podID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerID="30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2" exitCode=0
Sep 09 21:05:02.612909 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:02.612871    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" event={"ID":"f27f8f5e-6c53-400d-9890-a494ec75545a","Type":"ContainerDied","Data":"30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2"}
Sep 09 21:05:02.613391 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:02.613323    2571 scope.go:117] "RemoveContainer" containerID="30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2"
Sep 09 21:05:02.614651 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:02.614614    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 21:05:02.614737 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:02.614707    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 21:05:02.614841 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:02.614752    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 21:05:02.871357 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:02.871267    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:03.126988 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.126891    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:03.222379 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.222341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:03.279109 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.278817    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"]
Sep 09 21:05:03.283628 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.283598    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:03.285758 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.285686    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"]
Sep 09 21:05:03.286326 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.286297    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-98dad6\""
Sep 09 21:05:03.294824 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.294797    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"]
Sep 09 21:05:03.294950 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.294912    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:03.295708 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.295684    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"]
Sep 09 21:05:03.482835 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.482781    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs\" (UID: \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:03.483062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.482846    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj\" (UID: \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:03.483062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.482897    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvngp\" (UniqueName: \"kubernetes.io/projected/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-kube-api-access-gvngp\") pod \"payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs\" (UID: \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:03.483062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.482934    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv6wn\" (UniqueName: \"kubernetes.io/projected/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-kube-api-access-qv6wn\") pod \"payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj\" (UID: \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:03.489654 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.489570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:03.583415 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.583381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvngp\" (UniqueName: \"kubernetes.io/projected/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-kube-api-access-gvngp\") pod \"payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs\" (UID: \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:03.583415 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.583420    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv6wn\" (UniqueName: \"kubernetes.io/projected/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-kube-api-access-qv6wn\") pod \"payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj\" (UID: \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:03.583708 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.583543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs\" (UID: \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:03.583708 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.583569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj\" (UID: \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:03.584345 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.584324    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs\" (UID: \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:03.584414 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.584351    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj\" (UID: \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:03.591632 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.591598    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv6wn\" (UniqueName: \"kubernetes.io/projected/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-kube-api-access-qv6wn\") pod \"payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj\" (UID: \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:03.591757 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.591636    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvngp\" (UniqueName: \"kubernetes.io/projected/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-kube-api-access-gvngp\") pod \"payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs\" (UID: \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:03.598458 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.598432    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:03.608618 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.608593    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:03.623687 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.623656    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" event={"ID":"f27f8f5e-6c53-400d-9890-a494ec75545a","Type":"ContainerStarted","Data":"3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875"}
Sep 09 21:05:03.912303 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.912275    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"]
Sep 09 21:05:03.913442 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:05:03.913418    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8968d1_d9d8_4f94_b7a9_f3842cd3f6f1.slice/crio-6e8ce08af085e8fee05e0adecf435522dc38e54a4829b25a9ed7a7752183ef6c WatchSource:0}: Error finding container 6e8ce08af085e8fee05e0adecf435522dc38e54a4829b25a9ed7a7752183ef6c: Status 404 returned error can't find the container with id 6e8ce08af085e8fee05e0adecf435522dc38e54a4829b25a9ed7a7752183ef6c
Sep 09 21:05:03.920940 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:03.920917    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"]
Sep 09 21:05:03.922519 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:05:03.922425    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e42b09_9cd1_4f27_85c9_3c5b9ef219ca.slice/crio-a3b932bdb2744308099eb7d47916158572abeb018950dc62e14bfad132a62708 WatchSource:0}: Error finding container a3b932bdb2744308099eb7d47916158572abeb018950dc62e14bfad132a62708: Status 404 returned error can't find the container with id a3b932bdb2744308099eb7d47916158572abeb018950dc62e14bfad132a62708
Sep 09 21:05:04.158675 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.158638    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:05:04.158882 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.158687    2571 patch_prober.go:28] interesting pod/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 21:05:04.159071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.159039    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 21:05:04.631830 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.631792    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs" event={"ID":"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1","Type":"ContainerStarted","Data":"297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60"}
Sep 09 21:05:04.632363 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.632007    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:04.632363 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.632034    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs" event={"ID":"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1","Type":"ContainerStarted","Data":"6e8ce08af085e8fee05e0adecf435522dc38e54a4829b25a9ed7a7752183ef6c"}
Sep 09 21:05:04.633646 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.633617    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj" event={"ID":"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca","Type":"ContainerStarted","Data":"bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519"}
Sep 09 21:05:04.633788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.633669    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj" event={"ID":"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca","Type":"ContainerStarted","Data":"a3b932bdb2744308099eb7d47916158572abeb018950dc62e14bfad132a62708"}
Sep 09 21:05:04.633866 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.633848    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:04.701996 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.701931    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs" podStartSLOduration=1.7019122709999999 podStartE2EDuration="1.701912271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:05:04.678028988 +0000 UTC m=+998.648595707" watchObservedRunningTime="2026-09-09 21:05:04.701912271 +0000 UTC m=+998.672478989"
Sep 09 21:05:04.702578 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:04.702544    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj" podStartSLOduration=1.702534203 podStartE2EDuration="1.702534203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:05:04.699346416 +0000 UTC m=+998.669913134" watchObservedRunningTime="2026-09-09 21:05:04.702534203 +0000 UTC m=+998.673100974"
Sep 09 21:05:05.121879 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:05.121845    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:05.158445 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:05.158393    2571 patch_prober.go:28] interesting pod/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 21:05:05.158651 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:05.158479    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 21:05:05.229516 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:05.229468    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:05.326251 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:05.326141    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:05.478028 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:05.477984    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:06.158882 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:06.158829    2571 patch_prober.go:28] interesting pod/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 21:05:06.159531 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:06.158907    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 21:05:07.158553 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:07.158510    2571 patch_prober.go:28] interesting pod/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 21:05:07.158983 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:07.158947    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 21:05:08.159268 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:08.159223    2571 patch_prober.go:28] interesting pod/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Sep 09 21:05:08.159777 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:08.159299    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Sep 09 21:05:09.060112 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:09.060057    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:09.162484 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:09.162454    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:05:09.162919 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:09.162902    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:05:09.163657 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:09.163635    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:05:09.918117 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:09.918054    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:10.365019 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:10.364974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:10.912934 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:10.912902    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:11.466230 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:11.466192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:12.609279 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:12.609243    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:12.705645 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:12.705609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:14.242646 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.242609    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"]
Sep 09 21:05:14.252466 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.252438    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"
Sep 09 21:05:14.253716 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.253689    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"]
Sep 09 21:05:14.254769 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.254750    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:05:14.286295 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.286252    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68fb7e95-471b-4364-80bb-cecbe820ee05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql\" (UID: \"68fb7e95-471b-4364-80bb-cecbe820ee05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"
Sep 09 21:05:14.387835 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.387799    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68fb7e95-471b-4364-80bb-cecbe820ee05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql\" (UID: \"68fb7e95-471b-4364-80bb-cecbe820ee05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"
Sep 09 21:05:14.388470 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.388449    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68fb7e95-471b-4364-80bb-cecbe820ee05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql\" (UID: \"68fb7e95-471b-4364-80bb-cecbe820ee05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"
Sep 09 21:05:14.565263 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.565169    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"
Sep 09 21:05:14.840161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:14.840124    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"]
Sep 09 21:05:14.841254 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:05:14.841229    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68fb7e95_471b_4364_80bb_cecbe820ee05.slice/crio-11bf621834715e37562c145d389457b0642280ffdf1173e96bbdc9e79913bfc3 WatchSource:0}: Error finding container 11bf621834715e37562c145d389457b0642280ffdf1173e96bbdc9e79913bfc3: Status 404 returned error can't find the container with id 11bf621834715e37562c145d389457b0642280ffdf1173e96bbdc9e79913bfc3
Sep 09 21:05:15.642609 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:15.642579    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:05:15.643027 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:15.642647    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:05:15.716608 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:15.716560    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql" event={"ID":"68fb7e95-471b-4364-80bb-cecbe820ee05","Type":"ContainerStarted","Data":"11bf621834715e37562c145d389457b0642280ffdf1173e96bbdc9e79913bfc3"}
Sep 09 21:05:17.731536 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:17.731493    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql" event={"ID":"68fb7e95-471b-4364-80bb-cecbe820ee05","Type":"ContainerStarted","Data":"23c3f156a2a9050bcefc6ca8fdb30757dd1ee3d5e0fc8c3a162eeaec1120b209"}
Sep 09 21:05:17.754823 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:17.754772    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql" podStartSLOduration=1.524057928 podStartE2EDuration="3.754755272s" totalImagesPullingTime="2.230697344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:05:17.753057422 +0000 UTC m=+1011.723624139" watchObservedRunningTime="2026-09-09 21:05:17.754755272 +0000 UTC m=+1011.725321982"
Sep 09 21:05:18.739645 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:18.739604    2571 generic.go:356] "Generic (PLEG): container finished" podID="68fb7e95-471b-4364-80bb-cecbe820ee05" containerID="23c3f156a2a9050bcefc6ca8fdb30757dd1ee3d5e0fc8c3a162eeaec1120b209" exitCode=0
Sep 09 21:05:18.740114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:18.739675    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql" event={"ID":"68fb7e95-471b-4364-80bb-cecbe820ee05","Type":"ContainerDied","Data":"23c3f156a2a9050bcefc6ca8fdb30757dd1ee3d5e0fc8c3a162eeaec1120b209"}
Sep 09 21:05:19.888535 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:19.888508    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"
Sep 09 21:05:19.948921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:19.948886    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68fb7e95-471b-4364-80bb-cecbe820ee05-maas-api-service-ca\") pod \"68fb7e95-471b-4364-80bb-cecbe820ee05\" (UID: \"68fb7e95-471b-4364-80bb-cecbe820ee05\") "
Sep 09 21:05:19.949306 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:19.949282    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68fb7e95-471b-4364-80bb-cecbe820ee05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "68fb7e95-471b-4364-80bb-cecbe820ee05" (UID: "68fb7e95-471b-4364-80bb-cecbe820ee05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:05:20.050048 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:20.049964    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68fb7e95-471b-4364-80bb-cecbe820ee05-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:05:20.754859 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:20.754822    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"
Sep 09 21:05:20.755119 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:20.754898    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql" event={"ID":"68fb7e95-471b-4364-80bb-cecbe820ee05","Type":"ContainerDied","Data":"11bf621834715e37562c145d389457b0642280ffdf1173e96bbdc9e79913bfc3"}
Sep 09 21:05:20.755119 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:20.754944    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11bf621834715e37562c145d389457b0642280ffdf1173e96bbdc9e79913bfc3"
Sep 09 21:05:24.648089 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:24.648048    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:25.835889 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:25.835857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:25.998813 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:25.998776    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:26.087051 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.086962    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:26.188723 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.188685    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"]
Sep 09 21:05:26.191426 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.191403    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ab334d49-pn5ql"]
Sep 09 21:05:26.253028 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.252994    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:26.334536 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.334494    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:26.404533 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.404433    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:26.567015 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.566985    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68fb7e95-471b-4364-80bb-cecbe820ee05" path="/var/lib/kubelet/pods/68fb7e95-471b-4364-80bb-cecbe820ee05/volumes"
Sep 09 21:05:26.839293 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.839254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:26.941875 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:26.941834    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:27.056853 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:27.056811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:27.376529 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:27.376488    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:27.474593 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:27.474549    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:30.080199 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:30.080169    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:30.180268 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:30.180237    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:30.312579 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:30.312546    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:30.414014 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:30.413929    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:30.816568 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:30.816526    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:30.913345 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:30.913304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:31.013161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:31.013129    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:42.809579 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:42.809545    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:05:57.050489 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:05:57.050446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:08.048644 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:08.048610    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:08.149800 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:08.149759    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:11.584186 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:11.584141    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:11.654835 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:11.654798    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:11.754642 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:11.754612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:11.815861 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:11.815830    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:11.965116 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:11.965065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:12.100553 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:12.100520    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:12.381943 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:12.381844    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:12.694242 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:12.694200    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:12.901055 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:12.901007    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:15.158217 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:15.158184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:15.254959 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:15.254904    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:16.473556 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:16.473519    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:16.573266 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:16.573231    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:16.908468 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:16.908377    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:17.078692 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.078654    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:17.181875 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.181772    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:17.376970 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.376933    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"]
Sep 09 21:06:17.377436 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.377423    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68fb7e95-471b-4364-80bb-cecbe820ee05" containerName="revoke-keys"
Sep 09 21:06:17.377494 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.377438    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="68fb7e95-471b-4364-80bb-cecbe820ee05" containerName="revoke-keys"
Sep 09 21:06:17.377549 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.377540    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="68fb7e95-471b-4364-80bb-cecbe820ee05" containerName="revoke-keys"
Sep 09 21:06:17.380987 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.380958    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:17.384120 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.384099    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-857dd97d\""
Sep 09 21:06:17.384484 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.384467    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-857dd97d-dockercfg-sq68p\""
Sep 09 21:06:17.397470 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.397444    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"]
Sep 09 21:06:17.528768 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.528682    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2900ddb2-05c4-4fd3-a3c3-da47c419f699-plugins-config-volume\") pod \"payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t\" (UID: \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\") " pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:17.529225 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.528810    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t4zf\" (UniqueName: \"kubernetes.io/projected/2900ddb2-05c4-4fd3-a3c3-da47c419f699-kube-api-access-9t4zf\") pod \"payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t\" (UID: \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\") " pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:17.630318 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.630287    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2900ddb2-05c4-4fd3-a3c3-da47c419f699-plugins-config-volume\") pod \"payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t\" (UID: \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\") " pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:17.630552 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.630370    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t4zf\" (UniqueName: \"kubernetes.io/projected/2900ddb2-05c4-4fd3-a3c3-da47c419f699-kube-api-access-9t4zf\") pod \"payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t\" (UID: \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\") " pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:17.631311 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.631275    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2900ddb2-05c4-4fd3-a3c3-da47c419f699-plugins-config-volume\") pod \"payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t\" (UID: \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\") " pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:17.639158 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.639136    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t4zf\" (UniqueName: \"kubernetes.io/projected/2900ddb2-05c4-4fd3-a3c3-da47c419f699-kube-api-access-9t4zf\") pod \"payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t\" (UID: \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\") " pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:17.692069 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.692025    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:17.901860 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.901746    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:17.972023 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:17.971992    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"]
Sep 09 21:06:17.974105 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:06:17.974051    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2900ddb2_05c4_4fd3_a3c3_da47c419f699.slice/crio-7400d28bb3c017b74796603f9aa962d377d433a529d77a35bbd954edd82da2c9 WatchSource:0}: Error finding container 7400d28bb3c017b74796603f9aa962d377d433a529d77a35bbd954edd82da2c9: Status 404 returned error can't find the container with id 7400d28bb3c017b74796603f9aa962d377d433a529d77a35bbd954edd82da2c9
Sep 09 21:06:18.131422 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:18.131387    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t" event={"ID":"2900ddb2-05c4-4fd3-a3c3-da47c419f699","Type":"ContainerStarted","Data":"d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442"}
Sep 09 21:06:18.131621 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:18.131559    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:18.131621 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:18.131600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t" event={"ID":"2900ddb2-05c4-4fd3-a3c3-da47c419f699","Type":"ContainerStarted","Data":"7400d28bb3c017b74796603f9aa962d377d433a529d77a35bbd954edd82da2c9"}
Sep 09 21:06:18.154116 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:18.153960    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t" podStartSLOduration=1.15394259 podStartE2EDuration="1.15394259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:18.152054911 +0000 UTC m=+1072.122621628" watchObservedRunningTime="2026-09-09 21:06:18.15394259 +0000 UTC m=+1072.124509306"
Sep 09 21:06:29.139622 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:29.139592    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:45.338353 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:45.338314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:47.239812 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.239777    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:47.266862 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.266823    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"]
Sep 09 21:06:47.267106 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.267060    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t" podUID="2900ddb2-05c4-4fd3-a3c3-da47c419f699" containerName="payload-processing" containerID="cri-o://d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442" gracePeriod=30
Sep 09 21:06:47.337053 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.337016    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:47.435305 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.435270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:47.524550 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.524526    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:47.538376 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.538340    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:47.636200 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.636166    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2900ddb2-05c4-4fd3-a3c3-da47c419f699-plugins-config-volume\") pod \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\" (UID: \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\") "
Sep 09 21:06:47.636400 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.636302    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t4zf\" (UniqueName: \"kubernetes.io/projected/2900ddb2-05c4-4fd3-a3c3-da47c419f699-kube-api-access-9t4zf\") pod \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\" (UID: \"2900ddb2-05c4-4fd3-a3c3-da47c419f699\") "
Sep 09 21:06:47.636559 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.636533    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2900ddb2-05c4-4fd3-a3c3-da47c419f699-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2900ddb2-05c4-4fd3-a3c3-da47c419f699" (UID: "2900ddb2-05c4-4fd3-a3c3-da47c419f699"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:06:47.638623 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.638601    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2900ddb2-05c4-4fd3-a3c3-da47c419f699-kube-api-access-9t4zf" (OuterVolumeSpecName: "kube-api-access-9t4zf") pod "2900ddb2-05c4-4fd3-a3c3-da47c419f699" (UID: "2900ddb2-05c4-4fd3-a3c3-da47c419f699"). InnerVolumeSpecName "kube-api-access-9t4zf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:06:47.737418 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.737375    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t4zf\" (UniqueName: \"kubernetes.io/projected/2900ddb2-05c4-4fd3-a3c3-da47c419f699-kube-api-access-9t4zf\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:06:47.737418 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:47.737407    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2900ddb2-05c4-4fd3-a3c3-da47c419f699-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:06:48.328722 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.328686    2571 generic.go:356] "Generic (PLEG): container finished" podID="2900ddb2-05c4-4fd3-a3c3-da47c419f699" containerID="d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442" exitCode=0
Sep 09 21:06:48.329229 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.328740    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"
Sep 09 21:06:48.329229 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.328774    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t" event={"ID":"2900ddb2-05c4-4fd3-a3c3-da47c419f699","Type":"ContainerDied","Data":"d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442"}
Sep 09 21:06:48.329229 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.328825    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t" event={"ID":"2900ddb2-05c4-4fd3-a3c3-da47c419f699","Type":"ContainerDied","Data":"7400d28bb3c017b74796603f9aa962d377d433a529d77a35bbd954edd82da2c9"}
Sep 09 21:06:48.329229 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.328847    2571 scope.go:117] "RemoveContainer" containerID="d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442"
Sep 09 21:06:48.339316 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.339300    2571 scope.go:117] "RemoveContainer" containerID="d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442"
Sep 09 21:06:48.339583 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:06:48.339566    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442\": container with ID starting with d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442 not found: ID does not exist" containerID="d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442"
Sep 09 21:06:48.339645 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.339591    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442"} err="failed to get container status \"d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442\": rpc error: code = NotFound desc = could not find container \"d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442\": container with ID starting with d8c9c3653ab5dfde575e6dc80767366020829a3eaa08a36d8aea341713996442 not found: ID does not exist"
Sep 09 21:06:48.354339 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.354314    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"]
Sep 09 21:06:48.362746 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.362720    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-857dd97d-585956dcbc-5vq8t"]
Sep 09 21:06:48.578813 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.578709    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2900ddb2-05c4-4fd3-a3c3-da47c419f699" path="/var/lib/kubelet/pods/2900ddb2-05c4-4fd3-a3c3-da47c419f699/volumes"
Sep 09 21:06:48.597173 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.597140    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:48.841275 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:48.841189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:49.021811 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:49.021768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:49.324525 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:49.324466    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:49.453264 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:49.453229    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:50.028244 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:50.028211    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:52.111057 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.111017    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"]
Sep 09 21:06:52.111551 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.111519    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2900ddb2-05c4-4fd3-a3c3-da47c419f699" containerName="payload-processing"
Sep 09 21:06:52.111551 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.111530    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2900ddb2-05c4-4fd3-a3c3-da47c419f699" containerName="payload-processing"
Sep 09 21:06:52.111628 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.111610    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2900ddb2-05c4-4fd3-a3c3-da47c419f699" containerName="payload-processing"
Sep 09 21:06:52.116374 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.116348    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"
Sep 09 21:06:52.118607 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.118579    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:06:52.123270 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.123242    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"]
Sep 09 21:06:52.202955 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.202909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:52.281924 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.281886    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ae0806-0ee5-44b1-82a4-85aeac468a3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-80db9b-jwc64\" (UID: \"79ae0806-0ee5-44b1-82a4-85aeac468a3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"
Sep 09 21:06:52.306095 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.306044    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:52.383287 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.383193    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ae0806-0ee5-44b1-82a4-85aeac468a3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-80db9b-jwc64\" (UID: \"79ae0806-0ee5-44b1-82a4-85aeac468a3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"
Sep 09 21:06:52.383888 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.383866    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ae0806-0ee5-44b1-82a4-85aeac468a3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-80db9b-jwc64\" (UID: \"79ae0806-0ee5-44b1-82a4-85aeac468a3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"
Sep 09 21:06:52.429028 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.428982    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"
Sep 09 21:06:52.711789 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:52.711766    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"]
Sep 09 21:06:52.713301 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:06:52.713273    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ae0806_0ee5_44b1_82a4_85aeac468a3a.slice/crio-cad8e4d63bec0ee583950cb7f27713fb6f69c3858b9d2384ac23f574a20d8f65 WatchSource:0}: Error finding container cad8e4d63bec0ee583950cb7f27713fb6f69c3858b9d2384ac23f574a20d8f65: Status 404 returned error can't find the container with id cad8e4d63bec0ee583950cb7f27713fb6f69c3858b9d2384ac23f574a20d8f65
Sep 09 21:06:53.365246 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:53.365150    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64" event={"ID":"79ae0806-0ee5-44b1-82a4-85aeac468a3a","Type":"ContainerStarted","Data":"c720aa1af59ac87b964329dcf9aa6fd97dd004125591180a0a17cc2c63219315"}
Sep 09 21:06:53.365246 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:53.365202    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64" event={"ID":"79ae0806-0ee5-44b1-82a4-85aeac468a3a","Type":"ContainerStarted","Data":"cad8e4d63bec0ee583950cb7f27713fb6f69c3858b9d2384ac23f574a20d8f65"}
Sep 09 21:06:53.386876 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:53.386803    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64" podStartSLOduration=1.386779199 podStartE2EDuration="1.386779199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:53.381938891 +0000 UTC m=+1107.352505609" watchObservedRunningTime="2026-09-09 21:06:53.386779199 +0000 UTC m=+1107.357345917"
Sep 09 21:06:53.765933 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:53.765882    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:53.868682 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:53.868644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:53.985478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:53.985444    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:54.240850 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.240816    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:54.347866 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.347806    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:54.373087 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.373048    2571 generic.go:356] "Generic (PLEG): container finished" podID="79ae0806-0ee5-44b1-82a4-85aeac468a3a" containerID="c720aa1af59ac87b964329dcf9aa6fd97dd004125591180a0a17cc2c63219315" exitCode=0
Sep 09 21:06:54.373490 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.373120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64" event={"ID":"79ae0806-0ee5-44b1-82a4-85aeac468a3a","Type":"ContainerDied","Data":"c720aa1af59ac87b964329dcf9aa6fd97dd004125591180a0a17cc2c63219315"}
Sep 09 21:06:54.402238 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.402193    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"]
Sep 09 21:06:54.405932 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.405910    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:54.409389 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.409361    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-47d82e7b\""
Sep 09 21:06:54.409528 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.409443    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-47d82e7b-dockercfg-wn5n2\""
Sep 09 21:06:54.435764 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.435731    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"]
Sep 09 21:06:54.606703 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.606614    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w5h8\" (UniqueName: \"kubernetes.io/projected/7a62b8f3-ace4-4076-9202-27f1db3d011c-kube-api-access-9w5h8\") pod \"payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52\" (UID: \"7a62b8f3-ace4-4076-9202-27f1db3d011c\") " pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:54.606703 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.606652    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a62b8f3-ace4-4076-9202-27f1db3d011c-plugins-config-volume\") pod \"payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52\" (UID: \"7a62b8f3-ace4-4076-9202-27f1db3d011c\") " pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:54.707217 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.707179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w5h8\" (UniqueName: \"kubernetes.io/projected/7a62b8f3-ace4-4076-9202-27f1db3d011c-kube-api-access-9w5h8\") pod \"payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52\" (UID: \"7a62b8f3-ace4-4076-9202-27f1db3d011c\") " pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:54.707217 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.707219    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a62b8f3-ace4-4076-9202-27f1db3d011c-plugins-config-volume\") pod \"payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52\" (UID: \"7a62b8f3-ace4-4076-9202-27f1db3d011c\") " pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:54.707883 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.707866    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a62b8f3-ace4-4076-9202-27f1db3d011c-plugins-config-volume\") pod \"payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52\" (UID: \"7a62b8f3-ace4-4076-9202-27f1db3d011c\") " pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:54.715480 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.715445    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w5h8\" (UniqueName: \"kubernetes.io/projected/7a62b8f3-ace4-4076-9202-27f1db3d011c-kube-api-access-9w5h8\") pod \"payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52\" (UID: \"7a62b8f3-ace4-4076-9202-27f1db3d011c\") " pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:54.718230 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.718204    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:54.947757 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.947726    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:54.993731 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:54.993703    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"]
Sep 09 21:06:54.994874 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:06:54.994848    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a62b8f3_ace4_4076_9202_27f1db3d011c.slice/crio-586ca54cf24bcc98855ee48864e342a7ff61a033b8d25a410beb63948007b51e WatchSource:0}: Error finding container 586ca54cf24bcc98855ee48864e342a7ff61a033b8d25a410beb63948007b51e: Status 404 returned error can't find the container with id 586ca54cf24bcc98855ee48864e342a7ff61a033b8d25a410beb63948007b51e
Sep 09 21:06:55.382245 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.382153    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52" event={"ID":"7a62b8f3-ace4-4076-9202-27f1db3d011c","Type":"ContainerStarted","Data":"c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16"}
Sep 09 21:06:55.382245 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.382205    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52" event={"ID":"7a62b8f3-ace4-4076-9202-27f1db3d011c","Type":"ContainerStarted","Data":"586ca54cf24bcc98855ee48864e342a7ff61a033b8d25a410beb63948007b51e"}
Sep 09 21:06:55.382687 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.382387    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:06:55.397831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.397774    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52" podStartSLOduration=1.397753974 podStartE2EDuration="1.397753974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:06:55.397200206 +0000 UTC m=+1109.367766926" watchObservedRunningTime="2026-09-09 21:06:55.397753974 +0000 UTC m=+1109.368320692"
Sep 09 21:06:55.506825 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.506793    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"
Sep 09 21:06:55.513412 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.513384    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ae0806-0ee5-44b1-82a4-85aeac468a3a-maas-api-service-ca\") pod \"79ae0806-0ee5-44b1-82a4-85aeac468a3a\" (UID: \"79ae0806-0ee5-44b1-82a4-85aeac468a3a\") "
Sep 09 21:06:55.513718 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.513695    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79ae0806-0ee5-44b1-82a4-85aeac468a3a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "79ae0806-0ee5-44b1-82a4-85aeac468a3a" (UID: "79ae0806-0ee5-44b1-82a4-85aeac468a3a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:06:55.614568 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.614521    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/79ae0806-0ee5-44b1-82a4-85aeac468a3a-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:06:55.635919 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:55.635838    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:06:56.389902 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:56.389874    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"
Sep 09 21:06:56.390363 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:56.389914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64" event={"ID":"79ae0806-0ee5-44b1-82a4-85aeac468a3a","Type":"ContainerDied","Data":"cad8e4d63bec0ee583950cb7f27713fb6f69c3858b9d2384ac23f574a20d8f65"}
Sep 09 21:06:56.390363 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:06:56.389961    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cad8e4d63bec0ee583950cb7f27713fb6f69c3858b9d2384ac23f574a20d8f65"
Sep 09 21:07:02.353432 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:02.353393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:03.016885 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.016849    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"]
Sep 09 21:07:03.017222 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.017116    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l" podUID="61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de" containerName="payload-processing" containerID="cri-o://76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631" gracePeriod=30
Sep 09 21:07:03.024339 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.024310    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"]
Sep 09 21:07:03.024602 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.024539    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m" podUID="4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4" containerName="payload-pre-processing" containerID="cri-o://0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442" gracePeriod=30
Sep 09 21:07:03.047258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.047218    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:03.148118 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.148048    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:03.195448 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.194539    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"]
Sep 09 21:07:03.197907 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.197878    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-80db9b-jwc64"]
Sep 09 21:07:03.218058 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.218021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:03.311863 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.311837    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:07:03.315034 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.315011    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:07:03.387905 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.387872    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-plugins-config-volume\") pod \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\" (UID: \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\") "
Sep 09 21:07:03.388325 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.387922    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xl85q\" (UniqueName: \"kubernetes.io/projected/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-kube-api-access-xl85q\") pod \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\" (UID: \"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4\") "
Sep 09 21:07:03.388325 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.387950    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff5sp\" (UniqueName: \"kubernetes.io/projected/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-kube-api-access-ff5sp\") pod \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\" (UID: \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\") "
Sep 09 21:07:03.388325 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.387998    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-plugins-config-volume\") pod \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\" (UID: \"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de\") "
Sep 09 21:07:03.388442 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.388360    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4" (UID: "4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:03.388482 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.388437    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de" (UID: "61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:03.390395 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.390368    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-kube-api-access-xl85q" (OuterVolumeSpecName: "kube-api-access-xl85q") pod "4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4" (UID: "4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4"). InnerVolumeSpecName "kube-api-access-xl85q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:03.390509 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.390411    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-kube-api-access-ff5sp" (OuterVolumeSpecName: "kube-api-access-ff5sp") pod "61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de" (UID: "61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de"). InnerVolumeSpecName "kube-api-access-ff5sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:03.435605 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.435568    2571 generic.go:356] "Generic (PLEG): container finished" podID="4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4" containerID="0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442" exitCode=0
Sep 09 21:07:03.435778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.435635    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"
Sep 09 21:07:03.435778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.435660    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m" event={"ID":"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4","Type":"ContainerDied","Data":"0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442"}
Sep 09 21:07:03.435778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.435748    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m" event={"ID":"4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4","Type":"ContainerDied","Data":"4c032dcbb0fd7d00e269f7dbb48bc27f6f547a0abd59aa4257f5a2e68cbf4de9"}
Sep 09 21:07:03.435778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.435778    2571 scope.go:117] "RemoveContainer" containerID="0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442"
Sep 09 21:07:03.437222 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.437198    2571 generic.go:356] "Generic (PLEG): container finished" podID="61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de" containerID="76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631" exitCode=0
Sep 09 21:07:03.437328 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.437267    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"
Sep 09 21:07:03.437328 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.437278    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l" event={"ID":"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de","Type":"ContainerDied","Data":"76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631"}
Sep 09 21:07:03.437328 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.437318    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l" event={"ID":"61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de","Type":"ContainerDied","Data":"1d59ef88a5028bf94db39f55ef989b80b7f71229aeb6c222e2ed100ae0d57282"}
Sep 09 21:07:03.446913 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.446894    2571 scope.go:117] "RemoveContainer" containerID="0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442"
Sep 09 21:07:03.447309 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:07:03.447280    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442\": container with ID starting with 0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442 not found: ID does not exist" containerID="0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442"
Sep 09 21:07:03.447399 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.447321    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442"} err="failed to get container status \"0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442\": rpc error: code = NotFound desc = could not find container \"0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442\": container with ID starting with 0706344e37711e52aa7965d8af3959d7f52c9213e8561e2471a6945e94287442 not found: ID does not exist"
Sep 09 21:07:03.447399 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.447346    2571 scope.go:117] "RemoveContainer" containerID="76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631"
Sep 09 21:07:03.456522 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.456503    2571 scope.go:117] "RemoveContainer" containerID="76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631"
Sep 09 21:07:03.456820 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:07:03.456802    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631\": container with ID starting with 76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631 not found: ID does not exist" containerID="76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631"
Sep 09 21:07:03.456890 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.456826    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631"} err="failed to get container status \"76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631\": rpc error: code = NotFound desc = could not find container \"76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631\": container with ID starting with 76995c362386c9c33460e820e9117acf16c19076a225e4e80578442ae9d16631 not found: ID does not exist"
Sep 09 21:07:03.467977 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.467953    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"]
Sep 09 21:07:03.471576 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.471554    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-80db9b-57bfb8dbdb-wx76m"]
Sep 09 21:07:03.482848 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.482810    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"]
Sep 09 21:07:03.486277 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.486249    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-80db9b-5bbd7744c5-pgm9l"]
Sep 09 21:07:03.489884 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.489859    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:03.489987 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.489887    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:03.489987 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.489902    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl85q\" (UniqueName: \"kubernetes.io/projected/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4-kube-api-access-xl85q\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:03.489987 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.489914    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff5sp\" (UniqueName: \"kubernetes.io/projected/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de-kube-api-access-ff5sp\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:03.547237 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:03.547151    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:04.566881 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:04.566845    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4" path="/var/lib/kubelet/pods/4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4/volumes"
Sep 09 21:07:04.567317 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:04.567252    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de" path="/var/lib/kubelet/pods/61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de/volumes"
Sep 09 21:07:04.567554 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:04.567540    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79ae0806-0ee5-44b1-82a4-85aeac468a3a" path="/var/lib/kubelet/pods/79ae0806-0ee5-44b1-82a4-85aeac468a3a/volumes"
Sep 09 21:07:06.392006 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:06.391976    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:07:09.854582 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:09.854552    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:09.932743 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:09.932710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:10.097468 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:10.097428    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"]
Sep 09 21:07:10.097740 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:10.097712    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy" containerID="cri-o://3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875" gracePeriod=30
Sep 09 21:07:10.123142 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:10.123021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:10.546755 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:10.546705    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:10.849325 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:10.849241    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:11.393262 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:11.393224    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:14.030192 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:14.030151    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:14.213666 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:14.213629    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:14.881598 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:14.881542    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:15.385927 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.385897    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:07:15.515624 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515522    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4ggg\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-kube-api-access-q4ggg\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.515624 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515568    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-envoy\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.515624 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515619    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-credential-socket\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.515899 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515693    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-data\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.515899 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515721    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-podinfo\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.515899 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515758    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27f8f5e-6c53-400d-9890-a494ec75545a-istiod-ca-cert\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.515899 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515785    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-certs\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.515899 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515809    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-socket\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.515899 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.515844    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-token\") pod \"f27f8f5e-6c53-400d-9890-a494ec75545a\" (UID: \"f27f8f5e-6c53-400d-9890-a494ec75545a\") "
Sep 09 21:07:15.516206 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.516015    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:15.516206 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.516024    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:15.516359 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.516331    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f27f8f5e-6c53-400d-9890-a494ec75545a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:15.516418 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.516371    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-credential-socket\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.516418 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.516400    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-data\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.516626 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.516604    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:15.516688 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.516662    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:15.518694 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.518661    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 21:07:15.518843 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.518780    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-kube-api-access-q4ggg" (OuterVolumeSpecName: "kube-api-access-q4ggg") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "kube-api-access-q4ggg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:15.518902 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.518845    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:15.520160 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.520133    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f27f8f5e-6c53-400d-9890-a494ec75545a" (UID: "f27f8f5e-6c53-400d-9890-a494ec75545a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:07:15.526475 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.526443    2571 generic.go:356] "Generic (PLEG): container finished" podID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerID="3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875" exitCode=0
Sep 09 21:07:15.526673 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.526492    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"
Sep 09 21:07:15.526673 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.526543    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" event={"ID":"f27f8f5e-6c53-400d-9890-a494ec75545a","Type":"ContainerDied","Data":"3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875"}
Sep 09 21:07:15.526673 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.526656    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww" event={"ID":"f27f8f5e-6c53-400d-9890-a494ec75545a","Type":"ContainerDied","Data":"eb4a3146ce6d9878074314445d793d95c19d7856cb51a97023eb7650b4bb0a68"}
Sep 09 21:07:15.526831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.526692    2571 scope.go:117] "RemoveContainer" containerID="3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875"
Sep 09 21:07:15.551724 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.549327    2571 scope.go:117] "RemoveContainer" containerID="30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2"
Sep 09 21:07:15.560298 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.560271    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"]
Sep 09 21:07:15.564130 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.564112    2571 scope.go:117] "RemoveContainer" containerID="3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875"
Sep 09 21:07:15.564527 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:07:15.564505    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875\": container with ID starting with 3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875 not found: ID does not exist" containerID="3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875"
Sep 09 21:07:15.564606 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.564543    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875"} err="failed to get container status \"3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875\": rpc error: code = NotFound desc = could not find container \"3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875\": container with ID starting with 3a1945595182c7c92412d1081a7cb6d38820ca4de1c4fb5dd6a2f87e68438875 not found: ID does not exist"
Sep 09 21:07:15.564606 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.564571    2571 scope.go:117] "RemoveContainer" containerID="30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2"
Sep 09 21:07:15.564908 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:07:15.564891    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2\": container with ID starting with 30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2 not found: ID does not exist" containerID="30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2"
Sep 09 21:07:15.564965 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.564915    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2"} err="failed to get container status \"30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2\": rpc error: code = NotFound desc = could not find container \"30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2\": container with ID starting with 30707e4febb09090e7c0a27eb944949e1096df6c55e0a55e706901197680c2a2 not found: ID does not exist"
Sep 09 21:07:15.566596 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.566570    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-80db9b-openshift-default-86f7654d7b-srpww"]
Sep 09 21:07:15.617506 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.617467    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4ggg\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-kube-api-access-q4ggg\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.617506 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.617511    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-envoy\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.617731 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.617526    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-podinfo\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.617731 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.617538    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f27f8f5e-6c53-400d-9890-a494ec75545a-istiod-ca-cert\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.617731 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.617549    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-certs\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.617731 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.617563    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f27f8f5e-6c53-400d-9890-a494ec75545a-workload-socket\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.617731 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.617575    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f27f8f5e-6c53-400d-9890-a494ec75545a-istio-token\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:15.656176 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.656145    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:15.753746 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.753710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:15.900386 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:15.900300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:16.229204 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.229177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:16.328631 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.328594    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:16.355104 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355056    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"]
Sep 09 21:07:16.355727 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355701    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy"
Sep 09 21:07:16.355727 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355722    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy"
Sep 09 21:07:16.355727 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355730    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79ae0806-0ee5-44b1-82a4-85aeac468a3a" containerName="revoke-keys"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355736    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="79ae0806-0ee5-44b1-82a4-85aeac468a3a" containerName="revoke-keys"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355751    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4" containerName="payload-pre-processing"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355757    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4" containerName="payload-pre-processing"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355773    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de" containerName="payload-processing"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355778    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de" containerName="payload-processing"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355844    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355854    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355862    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a02e01f-0c6a-4c89-b3ae-8d6de3969ee4" containerName="payload-pre-processing"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355870    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="61511b0c-3fc1-4a3b-a1e5-fcb8bdd4b6de" containerName="payload-processing"
Sep 09 21:07:16.355916 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.355876    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="79ae0806-0ee5-44b1-82a4-85aeac468a3a" containerName="revoke-keys"
Sep 09 21:07:16.360479 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.360460    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:16.362737 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.362722    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-0066f8-dockercfg-5vz4c\""
Sep 09 21:07:16.362814 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.362722    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-0066f8\""
Sep 09 21:07:16.369925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.369903    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"]
Sep 09 21:07:16.372629 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.372605    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"]
Sep 09 21:07:16.373030 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.373017    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy"
Sep 09 21:07:16.373030 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.373030    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" containerName="istio-proxy"
Sep 09 21:07:16.376322 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.376301    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:16.383814 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.383790    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"]
Sep 09 21:07:16.425730 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.425690    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0b3320e-8e43-44af-b31d-8b7cafe04b95-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq\" (UID: \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:16.426216 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.425769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxpwd\" (UniqueName: \"kubernetes.io/projected/f0b3320e-8e43-44af-b31d-8b7cafe04b95-kube-api-access-xxpwd\") pod \"payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq\" (UID: \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:16.527332 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.527231    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0b3320e-8e43-44af-b31d-8b7cafe04b95-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq\" (UID: \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:16.527332 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.527314    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7952\" (UniqueName: \"kubernetes.io/projected/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-kube-api-access-v7952\") pod \"payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb\" (UID: \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\") " pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:16.527571 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.527353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-plugins-config-volume\") pod \"payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb\" (UID: \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\") " pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:16.527571 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.527388    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxpwd\" (UniqueName: \"kubernetes.io/projected/f0b3320e-8e43-44af-b31d-8b7cafe04b95-kube-api-access-xxpwd\") pod \"payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq\" (UID: \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:16.527969 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.527946    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0b3320e-8e43-44af-b31d-8b7cafe04b95-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq\" (UID: \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:16.535286 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.535261    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxpwd\" (UniqueName: \"kubernetes.io/projected/f0b3320e-8e43-44af-b31d-8b7cafe04b95-kube-api-access-xxpwd\") pod \"payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq\" (UID: \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:16.569849 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.569817    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f27f8f5e-6c53-400d-9890-a494ec75545a" path="/var/lib/kubelet/pods/f27f8f5e-6c53-400d-9890-a494ec75545a/volumes"
Sep 09 21:07:16.629061 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.629019    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7952\" (UniqueName: \"kubernetes.io/projected/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-kube-api-access-v7952\") pod \"payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb\" (UID: \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\") " pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:16.629061 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.629066    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-plugins-config-volume\") pod \"payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb\" (UID: \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\") " pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:16.629838 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.629819    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-plugins-config-volume\") pod \"payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb\" (UID: \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\") " pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:16.636946 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.636925    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7952\" (UniqueName: \"kubernetes.io/projected/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-kube-api-access-v7952\") pod \"payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb\" (UID: \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\") " pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:16.672536 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.672505    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:16.690216 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.690189    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:16.805528 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.805431    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:16.910465 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.910435    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"]
Sep 09 21:07:16.915246 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.915228    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"
Sep 09 21:07:16.917863 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.917839    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:07:16.921349 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.921320    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"]
Sep 09 21:07:16.957118 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.957089    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"]
Sep 09 21:07:16.958303 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:07:16.958273    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b3320e_8e43_44af_b31d_8b7cafe04b95.slice/crio-d59b856ae5c0fe6cc6e2b6b303bb59d021464fdae60f566b6ab37005457ab628 WatchSource:0}: Error finding container d59b856ae5c0fe6cc6e2b6b303bb59d021464fdae60f566b6ab37005457ab628: Status 404 returned error can't find the container with id d59b856ae5c0fe6cc6e2b6b303bb59d021464fdae60f566b6ab37005457ab628
Sep 09 21:07:16.983361 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:16.983334    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"]
Sep 09 21:07:16.994448 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:07:16.994414    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdcdf1bf_4dc7_4738_a9a0_cf8224b78965.slice/crio-8a53c55249d16de64e6b42488a61be3952282a6684af9e5bb35461770f5a123a WatchSource:0}: Error finding container 8a53c55249d16de64e6b42488a61be3952282a6684af9e5bb35461770f5a123a: Status 404 returned error can't find the container with id 8a53c55249d16de64e6b42488a61be3952282a6684af9e5bb35461770f5a123a
Sep 09 21:07:17.033376 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.033346    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0813dd3-7193-4b15-9863-d1479a14cdb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt\" (UID: \"c0813dd3-7193-4b15-9863-d1479a14cdb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"
Sep 09 21:07:17.134694 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.134599    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0813dd3-7193-4b15-9863-d1479a14cdb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt\" (UID: \"c0813dd3-7193-4b15-9863-d1479a14cdb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"
Sep 09 21:07:17.135450 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.135426    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0813dd3-7193-4b15-9863-d1479a14cdb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt\" (UID: \"c0813dd3-7193-4b15-9863-d1479a14cdb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"
Sep 09 21:07:17.228367 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.228317    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"
Sep 09 21:07:17.510808 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.510780    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"]
Sep 09 21:07:17.513014 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:07:17.512988    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0813dd3_7193_4b15_9863_d1479a14cdb9.slice/crio-dd68f88be7e9f65215a877e2275205a085c6db0498de4f8a3d8b8fd4aa603341 WatchSource:0}: Error finding container dd68f88be7e9f65215a877e2275205a085c6db0498de4f8a3d8b8fd4aa603341: Status 404 returned error can't find the container with id dd68f88be7e9f65215a877e2275205a085c6db0498de4f8a3d8b8fd4aa603341
Sep 09 21:07:17.542009 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.541982    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb" event={"ID":"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965","Type":"ContainerStarted","Data":"181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8"}
Sep 09 21:07:17.542161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.542042    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb" event={"ID":"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965","Type":"ContainerStarted","Data":"8a53c55249d16de64e6b42488a61be3952282a6684af9e5bb35461770f5a123a"}
Sep 09 21:07:17.542209 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.542184    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:17.543605 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.543587    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq" event={"ID":"f0b3320e-8e43-44af-b31d-8b7cafe04b95","Type":"ContainerStarted","Data":"3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd"}
Sep 09 21:07:17.543704 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.543630    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq" event={"ID":"f0b3320e-8e43-44af-b31d-8b7cafe04b95","Type":"ContainerStarted","Data":"d59b856ae5c0fe6cc6e2b6b303bb59d021464fdae60f566b6ab37005457ab628"}
Sep 09 21:07:17.543704 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.543670    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:17.544874 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.544825    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt" event={"ID":"c0813dd3-7193-4b15-9863-d1479a14cdb9","Type":"ContainerStarted","Data":"dd68f88be7e9f65215a877e2275205a085c6db0498de4f8a3d8b8fd4aa603341"}
Sep 09 21:07:17.567147 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.567096    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb" podStartSLOduration=1.567061421 podStartE2EDuration="1.567061421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:07:17.56206492 +0000 UTC m=+1131.532631642" watchObservedRunningTime="2026-09-09 21:07:17.567061421 +0000 UTC m=+1131.537628141"
Sep 09 21:07:17.583453 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.583402    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq" podStartSLOduration=1.5833873550000002 podStartE2EDuration="1.583387355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:07:17.58109733 +0000 UTC m=+1131.551664044" watchObservedRunningTime="2026-09-09 21:07:17.583387355 +0000 UTC m=+1131.553954069"
Sep 09 21:07:17.693656 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:17.693616    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:18.553799 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:18.553764    2571 generic.go:356] "Generic (PLEG): container finished" podID="c0813dd3-7193-4b15-9863-d1479a14cdb9" containerID="541c806667720be5edce6e439b5c2050a017345ace200b9df6ef77abb15b0ae5" exitCode=0
Sep 09 21:07:18.554380 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:18.553812    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt" event={"ID":"c0813dd3-7193-4b15-9863-d1479a14cdb9","Type":"ContainerDied","Data":"541c806667720be5edce6e439b5c2050a017345ace200b9df6ef77abb15b0ae5"}
Sep 09 21:07:18.586097 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:18.586052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:19.696205 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:19.696182    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"
Sep 09 21:07:19.866201 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:19.866059    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0813dd3-7193-4b15-9863-d1479a14cdb9-maas-api-service-ca\") pod \"c0813dd3-7193-4b15-9863-d1479a14cdb9\" (UID: \"c0813dd3-7193-4b15-9863-d1479a14cdb9\") "
Sep 09 21:07:19.866413 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:19.866365    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0813dd3-7193-4b15-9863-d1479a14cdb9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0813dd3-7193-4b15-9863-d1479a14cdb9" (UID: "c0813dd3-7193-4b15-9863-d1479a14cdb9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:19.967820 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:19.967784    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0813dd3-7193-4b15-9863-d1479a14cdb9-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:20.568027 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:20.568001    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"
Sep 09 21:07:20.568924 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:20.568890    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt" event={"ID":"c0813dd3-7193-4b15-9863-d1479a14cdb9","Type":"ContainerDied","Data":"dd68f88be7e9f65215a877e2275205a085c6db0498de4f8a3d8b8fd4aa603341"}
Sep 09 21:07:20.569051 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:20.568933    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd68f88be7e9f65215a877e2275205a085c6db0498de4f8a3d8b8fd4aa603341"
Sep 09 21:07:23.936753 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:23.936711    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:23.970292 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:23.970259    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"]
Sep 09 21:07:23.970544 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:23.970517    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52" podUID="7a62b8f3-ace4-4076-9202-27f1db3d011c" containerName="payload-processing" containerID="cri-o://c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16" gracePeriod=30
Sep 09 21:07:24.038487 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.038437    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:24.139485 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.139450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:24.175696 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.174801    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"]
Sep 09 21:07:24.184140 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.184100    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-47d82e7b-5l5jt"]
Sep 09 21:07:24.210747 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.210706    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:24.238615 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.238587    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:07:24.349424 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.349392    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:24.412319 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.412285    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w5h8\" (UniqueName: \"kubernetes.io/projected/7a62b8f3-ace4-4076-9202-27f1db3d011c-kube-api-access-9w5h8\") pod \"7a62b8f3-ace4-4076-9202-27f1db3d011c\" (UID: \"7a62b8f3-ace4-4076-9202-27f1db3d011c\") "
Sep 09 21:07:24.412514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.412384    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a62b8f3-ace4-4076-9202-27f1db3d011c-plugins-config-volume\") pod \"7a62b8f3-ace4-4076-9202-27f1db3d011c\" (UID: \"7a62b8f3-ace4-4076-9202-27f1db3d011c\") "
Sep 09 21:07:24.412790 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.412762    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a62b8f3-ace4-4076-9202-27f1db3d011c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a62b8f3-ace4-4076-9202-27f1db3d011c" (UID: "7a62b8f3-ace4-4076-9202-27f1db3d011c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:07:24.414609 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.414581    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a62b8f3-ace4-4076-9202-27f1db3d011c-kube-api-access-9w5h8" (OuterVolumeSpecName: "kube-api-access-9w5h8") pod "7a62b8f3-ace4-4076-9202-27f1db3d011c" (UID: "7a62b8f3-ace4-4076-9202-27f1db3d011c"). InnerVolumeSpecName "kube-api-access-9w5h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:07:24.513862 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.513763    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a62b8f3-ace4-4076-9202-27f1db3d011c-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:24.513862 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.513798    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w5h8\" (UniqueName: \"kubernetes.io/projected/7a62b8f3-ace4-4076-9202-27f1db3d011c-kube-api-access-9w5h8\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:07:24.567585 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.567549    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0813dd3-7193-4b15-9863-d1479a14cdb9" path="/var/lib/kubelet/pods/c0813dd3-7193-4b15-9863-d1479a14cdb9/volumes"
Sep 09 21:07:24.594462 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.594419    2571 generic.go:356] "Generic (PLEG): container finished" podID="7a62b8f3-ace4-4076-9202-27f1db3d011c" containerID="c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16" exitCode=0
Sep 09 21:07:24.594653 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.594467    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"
Sep 09 21:07:24.594653 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.594501    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52" event={"ID":"7a62b8f3-ace4-4076-9202-27f1db3d011c","Type":"ContainerDied","Data":"c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16"}
Sep 09 21:07:24.594653 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.594563    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52" event={"ID":"7a62b8f3-ace4-4076-9202-27f1db3d011c","Type":"ContainerDied","Data":"586ca54cf24bcc98855ee48864e342a7ff61a033b8d25a410beb63948007b51e"}
Sep 09 21:07:24.594653 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.594591    2571 scope.go:117] "RemoveContainer" containerID="c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16"
Sep 09 21:07:24.607405 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.607376    2571 scope.go:117] "RemoveContainer" containerID="c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16"
Sep 09 21:07:24.608016 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:07:24.607993    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16\": container with ID starting with c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16 not found: ID does not exist" containerID="c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16"
Sep 09 21:07:24.608148 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.608024    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16"} err="failed to get container status \"c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16\": rpc error: code = NotFound desc = could not find container \"c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16\": container with ID starting with c349c0d04dbeeb403793041781b5f27c53514c579a9801483e28882d05ae2e16 not found: ID does not exist"
Sep 09 21:07:24.611651 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.611630    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"]
Sep 09 21:07:24.617541 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.617520    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-47d82e7b-7b7d8c7489-rhb52"]
Sep 09 21:07:24.982718 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:24.982685    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:25.082676 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:25.082642    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:25.208610 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:25.208573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:26.332214 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:26.332179    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:26.432509 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:26.432478    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:26.539857 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:26.539821    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:26.568173 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:26.568143    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a62b8f3-ace4-4076-9202-27f1db3d011c" path="/var/lib/kubelet/pods/7a62b8f3-ace4-4076-9202-27f1db3d011c/volumes"
Sep 09 21:07:27.311641 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:27.311606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:28.556733 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:28.556706    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:07:28.557186 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:28.557101    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:07:32.678997 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:32.678957    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:32.741309 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:32.741276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:32.828013 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:32.827975    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:32.930852 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:32.930745    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:33.025250 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:33.025212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:33.350644 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:33.350606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:36.656095 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:36.656045    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:38.982349 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:38.982315    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:43.028361 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:43.028330    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:43.088759 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:43.088714    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:07:59.290712 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:07:59.290616    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:02.229889 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:02.229848    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:05.186811 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:05.186772    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:05.244773 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:05.244737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:05.340303 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:05.340204    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:05.434281 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:05.434240    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:10.915536 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:10.915469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:11.026443 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.026401    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"]
Sep 09 21:08:11.026890 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.026862    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a62b8f3-ace4-4076-9202-27f1db3d011c" containerName="payload-processing"
Sep 09 21:08:11.026890 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.026880    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a62b8f3-ace4-4076-9202-27f1db3d011c" containerName="payload-processing"
Sep 09 21:08:11.027114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.026911    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0813dd3-7193-4b15-9863-d1479a14cdb9" containerName="revoke-keys"
Sep 09 21:08:11.027114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.026919    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0813dd3-7193-4b15-9863-d1479a14cdb9" containerName="revoke-keys"
Sep 09 21:08:11.027114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.027038    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0813dd3-7193-4b15-9863-d1479a14cdb9" containerName="revoke-keys"
Sep 09 21:08:11.027114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.027053    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a62b8f3-ace4-4076-9202-27f1db3d011c" containerName="payload-processing"
Sep 09 21:08:11.030497 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.030478    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"
Sep 09 21:08:11.033183 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.033154    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:08:11.037943 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.037916    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"]
Sep 09 21:08:11.167972 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.167871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc662b-3162-48b8-a319-2f34951bd0fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-0066f8-clkqm\" (UID: \"54cc662b-3162-48b8-a319-2f34951bd0fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"
Sep 09 21:08:11.268760 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.268720    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc662b-3162-48b8-a319-2f34951bd0fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-0066f8-clkqm\" (UID: \"54cc662b-3162-48b8-a319-2f34951bd0fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"
Sep 09 21:08:11.269406 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.269385    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc662b-3162-48b8-a319-2f34951bd0fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-0066f8-clkqm\" (UID: \"54cc662b-3162-48b8-a319-2f34951bd0fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"
Sep 09 21:08:11.343656 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.343620    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"
Sep 09 21:08:11.620773 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.620743    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"]
Sep 09 21:08:11.622877 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:08:11.622851    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54cc662b_3162_48b8_a319_2f34951bd0fb.slice/crio-76eabfd21874601f1885052f29472c65c4546eeed0f24be6f73c5807ef7a05a8 WatchSource:0}: Error finding container 76eabfd21874601f1885052f29472c65c4546eeed0f24be6f73c5807ef7a05a8: Status 404 returned error can't find the container with id 76eabfd21874601f1885052f29472c65c4546eeed0f24be6f73c5807ef7a05a8
Sep 09 21:08:11.882503 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:11.882474    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm" event={"ID":"54cc662b-3162-48b8-a319-2f34951bd0fb","Type":"ContainerStarted","Data":"76eabfd21874601f1885052f29472c65c4546eeed0f24be6f73c5807ef7a05a8"}
Sep 09 21:08:12.890608 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:12.890570    2571 generic.go:356] "Generic (PLEG): container finished" podID="54cc662b-3162-48b8-a319-2f34951bd0fb" containerID="9a959993ec93388cc1774b1ede42a1995bdb0081d3b51b3addbfb617ffce8b3d" exitCode=0
Sep 09 21:08:12.891015 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:12.890664    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm" event={"ID":"54cc662b-3162-48b8-a319-2f34951bd0fb","Type":"ContainerDied","Data":"9a959993ec93388cc1774b1ede42a1995bdb0081d3b51b3addbfb617ffce8b3d"}
Sep 09 21:08:14.034090 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:14.034045    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"
Sep 09 21:08:14.089411 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:14.089380    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc662b-3162-48b8-a319-2f34951bd0fb-maas-api-service-ca\") pod \"54cc662b-3162-48b8-a319-2f34951bd0fb\" (UID: \"54cc662b-3162-48b8-a319-2f34951bd0fb\") "
Sep 09 21:08:14.089730 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:14.089710    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54cc662b-3162-48b8-a319-2f34951bd0fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54cc662b-3162-48b8-a319-2f34951bd0fb" (UID: "54cc662b-3162-48b8-a319-2f34951bd0fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:14.190178 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:14.190043    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54cc662b-3162-48b8-a319-2f34951bd0fb-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:08:14.904431 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:14.904346    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"
Sep 09 21:08:14.904431 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:14.904381    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm" event={"ID":"54cc662b-3162-48b8-a319-2f34951bd0fb","Type":"ContainerDied","Data":"76eabfd21874601f1885052f29472c65c4546eeed0f24be6f73c5807ef7a05a8"}
Sep 09 21:08:14.904431 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:14.904428    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76eabfd21874601f1885052f29472c65c4546eeed0f24be6f73c5807ef7a05a8"
Sep 09 21:08:21.289823 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.289794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:21.592305 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.592197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:21.634946 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.634896    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"]
Sep 09 21:08:21.635234 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.635189    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb" podUID="cdcdf1bf-4dc7-4738-a9a0-cf8224b78965" containerName="payload-processing" containerID="cri-o://181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8" gracePeriod=30
Sep 09 21:08:21.638135 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.638106    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"]
Sep 09 21:08:21.638447 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.638411    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq" podUID="f0b3320e-8e43-44af-b31d-8b7cafe04b95" containerName="payload-pre-processing" containerID="cri-o://3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd" gracePeriod=30
Sep 09 21:08:21.658109 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.658034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:21.721979 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.721932    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:21.793277 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.793211    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"]
Sep 09 21:08:21.794411 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.794358    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:21.797114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.797092    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-0066f8-clkqm"]
Sep 09 21:08:21.859371 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.859340    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:21.918374 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.918348    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:08:21.921318 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.921301    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:08:21.950656 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.950629    2571 generic.go:356] "Generic (PLEG): container finished" podID="cdcdf1bf-4dc7-4738-a9a0-cf8224b78965" containerID="181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8" exitCode=0
Sep 09 21:08:21.950838 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.950669    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"
Sep 09 21:08:21.950838 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.950709    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb" event={"ID":"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965","Type":"ContainerDied","Data":"181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8"}
Sep 09 21:08:21.950838 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.950759    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb" event={"ID":"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965","Type":"ContainerDied","Data":"8a53c55249d16de64e6b42488a61be3952282a6684af9e5bb35461770f5a123a"}
Sep 09 21:08:21.950838 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.950790    2571 scope.go:117] "RemoveContainer" containerID="181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8"
Sep 09 21:08:21.952206 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.952167    2571 generic.go:356] "Generic (PLEG): container finished" podID="f0b3320e-8e43-44af-b31d-8b7cafe04b95" containerID="3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd" exitCode=0
Sep 09 21:08:21.952206 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.952202    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq" event={"ID":"f0b3320e-8e43-44af-b31d-8b7cafe04b95","Type":"ContainerDied","Data":"3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd"}
Sep 09 21:08:21.952354 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.952218    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"
Sep 09 21:08:21.952354 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.952240    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq" event={"ID":"f0b3320e-8e43-44af-b31d-8b7cafe04b95","Type":"ContainerDied","Data":"d59b856ae5c0fe6cc6e2b6b303bb59d021464fdae60f566b6ab37005457ab628"}
Sep 09 21:08:21.958555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.958528    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7952\" (UniqueName: \"kubernetes.io/projected/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-kube-api-access-v7952\") pod \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\" (UID: \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\") "
Sep 09 21:08:21.958676 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.958578    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxpwd\" (UniqueName: \"kubernetes.io/projected/f0b3320e-8e43-44af-b31d-8b7cafe04b95-kube-api-access-xxpwd\") pod \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\" (UID: \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\") "
Sep 09 21:08:21.958676 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.958652    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-plugins-config-volume\") pod \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\" (UID: \"cdcdf1bf-4dc7-4738-a9a0-cf8224b78965\") "
Sep 09 21:08:21.958810 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.958699    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0b3320e-8e43-44af-b31d-8b7cafe04b95-plugins-config-volume\") pod \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\" (UID: \"f0b3320e-8e43-44af-b31d-8b7cafe04b95\") "
Sep 09 21:08:21.959383 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.959195    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdcdf1bf-4dc7-4738-a9a0-cf8224b78965" (UID: "cdcdf1bf-4dc7-4738-a9a0-cf8224b78965"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:21.959383 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.959328    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0b3320e-8e43-44af-b31d-8b7cafe04b95-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0b3320e-8e43-44af-b31d-8b7cafe04b95" (UID: "f0b3320e-8e43-44af-b31d-8b7cafe04b95"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:08:21.961794 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.961758    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0b3320e-8e43-44af-b31d-8b7cafe04b95-kube-api-access-xxpwd" (OuterVolumeSpecName: "kube-api-access-xxpwd") pod "f0b3320e-8e43-44af-b31d-8b7cafe04b95" (UID: "f0b3320e-8e43-44af-b31d-8b7cafe04b95"). InnerVolumeSpecName "kube-api-access-xxpwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:21.961794 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.961782    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-kube-api-access-v7952" (OuterVolumeSpecName: "kube-api-access-v7952") pod "cdcdf1bf-4dc7-4738-a9a0-cf8224b78965" (UID: "cdcdf1bf-4dc7-4738-a9a0-cf8224b78965"). InnerVolumeSpecName "kube-api-access-v7952". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:08:21.964400 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.964359    2571 scope.go:117] "RemoveContainer" containerID="181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8"
Sep 09 21:08:21.964776 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:08:21.964754    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8\": container with ID starting with 181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8 not found: ID does not exist" containerID="181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8"
Sep 09 21:08:21.964872 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.964782    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8"} err="failed to get container status \"181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8\": rpc error: code = NotFound desc = could not find container \"181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8\": container with ID starting with 181c09da45f35a16f239c0dfb63d69d9266f7dfe3229f479f58732a10c8b30c8 not found: ID does not exist"
Sep 09 21:08:21.964872 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.964796    2571 scope.go:117] "RemoveContainer" containerID="3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd"
Sep 09 21:08:21.975581 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.975559    2571 scope.go:117] "RemoveContainer" containerID="3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd"
Sep 09 21:08:21.975873 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:08:21.975853    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd\": container with ID starting with 3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd not found: ID does not exist" containerID="3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd"
Sep 09 21:08:21.975980 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:21.975876    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd"} err="failed to get container status \"3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd\": rpc error: code = NotFound desc = could not find container \"3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd\": container with ID starting with 3a8974b5be2c845aaf8fb530f31668b00ba1100f405c69dec8e91825ab05c8cd not found: ID does not exist"
Sep 09 21:08:22.059633 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.059595    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:08:22.059633 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.059628    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0b3320e-8e43-44af-b31d-8b7cafe04b95-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:08:22.059633 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.059640    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7952\" (UniqueName: \"kubernetes.io/projected/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965-kube-api-access-v7952\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:08:22.059873 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.059650    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxpwd\" (UniqueName: \"kubernetes.io/projected/f0b3320e-8e43-44af-b31d-8b7cafe04b95-kube-api-access-xxpwd\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:08:22.173689 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.173592    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:22.278083 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.278057    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"]
Sep 09 21:08:22.281780 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.281759    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-0066f8-7d49cf4857-42qgb"]
Sep 09 21:08:22.292063 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.292036    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"]
Sep 09 21:08:22.296430 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.296411    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-0066f8-8688dcd98-2fslq"]
Sep 09 21:08:22.566368 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.566335    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54cc662b-3162-48b8-a319-2f34951bd0fb" path="/var/lib/kubelet/pods/54cc662b-3162-48b8-a319-2f34951bd0fb/volumes"
Sep 09 21:08:22.566725 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.566710    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdcdf1bf-4dc7-4738-a9a0-cf8224b78965" path="/var/lib/kubelet/pods/cdcdf1bf-4dc7-4738-a9a0-cf8224b78965/volumes"
Sep 09 21:08:22.567037 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:22.567026    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0b3320e-8e43-44af-b31d-8b7cafe04b95" path="/var/lib/kubelet/pods/f0b3320e-8e43-44af-b31d-8b7cafe04b95/volumes"
Sep 09 21:08:27.758007 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:27.757965    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:30.285842 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:30.285802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:32.019714 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:32.019672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:32.119818 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:32.119786    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:37.402877 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:37.402836    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:38.479033 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:38.478994    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:39.296326 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:39.296287    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:45.251193 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:45.251153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:45.364616 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:45.364578    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:45.453304 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:45.453262    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:45.568663 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:45.568574    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:46.247761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:46.247704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:46.346572 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:46.346538    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:46.442439 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:46.442388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:48.035396 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:48.035358    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:52.025653 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:52.025611    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:53.990839 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:53.990804    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:54.055089 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:54.055052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:54.671256 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:54.671223    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:55.227128 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:55.227095    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:55.355164 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:55.355132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:55.602223 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:55.602128    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:55.713155 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:55.713112    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:56.533823 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:56.533768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:08:56.837616 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:08:56.837514    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:00.692410 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:00.692375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:00.785150 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:00.785116    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:00.938381 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:00.938338    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:01.291185 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:01.291148    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:01.565950 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:01.565861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:04.623305 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:04.623273    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:04.725113 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:04.725056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:05.839989 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:05.839944    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:05.918044 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:05.918002    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:06.047368 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.047331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:06.332057 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.332021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:06.431502 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.431453    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:06.489143 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489096    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"]
Sep 09 21:09:06.489572 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489559    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0b3320e-8e43-44af-b31d-8b7cafe04b95" containerName="payload-pre-processing"
Sep 09 21:09:06.489620 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489574    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0b3320e-8e43-44af-b31d-8b7cafe04b95" containerName="payload-pre-processing"
Sep 09 21:09:06.489620 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489587    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdcdf1bf-4dc7-4738-a9a0-cf8224b78965" containerName="payload-processing"
Sep 09 21:09:06.489620 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489594    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdcdf1bf-4dc7-4738-a9a0-cf8224b78965" containerName="payload-processing"
Sep 09 21:09:06.489620 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489602    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54cc662b-3162-48b8-a319-2f34951bd0fb" containerName="revoke-keys"
Sep 09 21:09:06.489620 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489607    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="54cc662b-3162-48b8-a319-2f34951bd0fb" containerName="revoke-keys"
Sep 09 21:09:06.489778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489684    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0b3320e-8e43-44af-b31d-8b7cafe04b95" containerName="payload-pre-processing"
Sep 09 21:09:06.489778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489692    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdcdf1bf-4dc7-4738-a9a0-cf8224b78965" containerName="payload-processing"
Sep 09 21:09:06.489778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.489698    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="54cc662b-3162-48b8-a319-2f34951bd0fb" containerName="revoke-keys"
Sep 09 21:09:06.493132 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.493111    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:06.495945 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.495923    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-57783aa9\""
Sep 09 21:09:06.496057 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.495969    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-57783aa9-dockercfg-nz52f\""
Sep 09 21:09:06.504630 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.504598    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"]
Sep 09 21:09:06.509510 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.509486    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"]
Sep 09 21:09:06.513285 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.513268    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:06.524041 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.524010    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"]
Sep 09 21:09:06.592187 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.592100    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txhwl\" (UniqueName: \"kubernetes.io/projected/9522241c-da90-45d2-a198-5618a7d8e1ad-kube-api-access-txhwl\") pod \"payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv\" (UID: \"9522241c-da90-45d2-a198-5618a7d8e1ad\") " pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:06.592343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.592204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9522241c-da90-45d2-a198-5618a7d8e1ad-plugins-config-volume\") pod \"payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv\" (UID: \"9522241c-da90-45d2-a198-5618a7d8e1ad\") " pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:06.592343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.592256    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpcqh\" (UniqueName: \"kubernetes.io/projected/82521c56-a08f-40e5-99e0-4915cf317d5b-kube-api-access-cpcqh\") pod \"payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz\" (UID: \"82521c56-a08f-40e5-99e0-4915cf317d5b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:06.592343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.592321    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82521c56-a08f-40e5-99e0-4915cf317d5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz\" (UID: \"82521c56-a08f-40e5-99e0-4915cf317d5b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:06.693382 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.693337    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82521c56-a08f-40e5-99e0-4915cf317d5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz\" (UID: \"82521c56-a08f-40e5-99e0-4915cf317d5b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:06.693382 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.693395    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txhwl\" (UniqueName: \"kubernetes.io/projected/9522241c-da90-45d2-a198-5618a7d8e1ad-kube-api-access-txhwl\") pod \"payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv\" (UID: \"9522241c-da90-45d2-a198-5618a7d8e1ad\") " pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:06.693625 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.693445    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9522241c-da90-45d2-a198-5618a7d8e1ad-plugins-config-volume\") pod \"payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv\" (UID: \"9522241c-da90-45d2-a198-5618a7d8e1ad\") " pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:06.693625 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.693465    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpcqh\" (UniqueName: \"kubernetes.io/projected/82521c56-a08f-40e5-99e0-4915cf317d5b-kube-api-access-cpcqh\") pod \"payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz\" (UID: \"82521c56-a08f-40e5-99e0-4915cf317d5b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:06.694112 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.694091    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82521c56-a08f-40e5-99e0-4915cf317d5b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz\" (UID: \"82521c56-a08f-40e5-99e0-4915cf317d5b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:06.694324 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.694305    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9522241c-da90-45d2-a198-5618a7d8e1ad-plugins-config-volume\") pod \"payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv\" (UID: \"9522241c-da90-45d2-a198-5618a7d8e1ad\") " pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:06.701452 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.701426    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpcqh\" (UniqueName: \"kubernetes.io/projected/82521c56-a08f-40e5-99e0-4915cf317d5b-kube-api-access-cpcqh\") pod \"payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz\" (UID: \"82521c56-a08f-40e5-99e0-4915cf317d5b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:06.701650 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.701631    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txhwl\" (UniqueName: \"kubernetes.io/projected/9522241c-da90-45d2-a198-5618a7d8e1ad-kube-api-access-txhwl\") pod \"payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv\" (UID: \"9522241c-da90-45d2-a198-5618a7d8e1ad\") " pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:06.805827 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.805789    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:06.825539 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:06.825515    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:07.046923 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.046889    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"]
Sep 09 21:09:07.047434 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.047281    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp" podUID="30dca177-8dab-4d50-9d08-91b3e7da1018" containerName="maas-api" containerID="cri-o://0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933" gracePeriod=30
Sep 09 21:09:07.083639 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.083602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:07.085578 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.085555    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"]
Sep 09 21:09:07.086602 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:09:07.086573    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9522241c_da90_45d2_a198_5618a7d8e1ad.slice/crio-0fbe6c73fde67299831b62293ec2b0a95a44d3f0ce8347df49a03e1de6020de9 WatchSource:0}: Error finding container 0fbe6c73fde67299831b62293ec2b0a95a44d3f0ce8347df49a03e1de6020de9: Status 404 returned error can't find the container with id 0fbe6c73fde67299831b62293ec2b0a95a44d3f0ce8347df49a03e1de6020de9
Sep 09 21:09:07.119925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.119882    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"]
Sep 09 21:09:07.138602 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:09:07.138571    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82521c56_a08f_40e5_99e0_4915cf317d5b.slice/crio-6e5480b7e3b52d47759c3b43f27d6998e371c9fa8d8a9011bb63b2cf1e06c5ee WatchSource:0}: Error finding container 6e5480b7e3b52d47759c3b43f27d6998e371c9fa8d8a9011bb63b2cf1e06c5ee: Status 404 returned error can't find the container with id 6e5480b7e3b52d47759c3b43f27d6998e371c9fa8d8a9011bb63b2cf1e06c5ee
Sep 09 21:09:07.180595 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.180552    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:07.231468 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.231370    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv" event={"ID":"9522241c-da90-45d2-a198-5618a7d8e1ad","Type":"ContainerStarted","Data":"10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1"}
Sep 09 21:09:07.231468 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.231432    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv" event={"ID":"9522241c-da90-45d2-a198-5618a7d8e1ad","Type":"ContainerStarted","Data":"0fbe6c73fde67299831b62293ec2b0a95a44d3f0ce8347df49a03e1de6020de9"}
Sep 09 21:09:07.231723 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.231571    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:07.233271 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.233230    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz" event={"ID":"82521c56-a08f-40e5-99e0-4915cf317d5b","Type":"ContainerStarted","Data":"ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7"}
Sep 09 21:09:07.233390 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.233282    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:07.233390 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.233313    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz" event={"ID":"82521c56-a08f-40e5-99e0-4915cf317d5b","Type":"ContainerStarted","Data":"6e5480b7e3b52d47759c3b43f27d6998e371c9fa8d8a9011bb63b2cf1e06c5ee"}
Sep 09 21:09:07.257108 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.257008    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv" podStartSLOduration=1.256985192 podStartE2EDuration="1.256985192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:09:07.254627274 +0000 UTC m=+1241.225193994" watchObservedRunningTime="2026-09-09 21:09:07.256985192 +0000 UTC m=+1241.227551909"
Sep 09 21:09:07.276003 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.275946    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz" podStartSLOduration=1.275930244 podStartE2EDuration="1.275930244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:09:07.274163259 +0000 UTC m=+1241.244729973" watchObservedRunningTime="2026-09-09 21:09:07.275930244 +0000 UTC m=+1241.246496984"
Sep 09 21:09:07.280439 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.280401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:07.364176 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.364136    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:07.489230 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.489205    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:09:07.602038 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.601992    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls\") pod \"30dca177-8dab-4d50-9d08-91b3e7da1018\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") "
Sep 09 21:09:07.602258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.602064    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30dca177-8dab-4d50-9d08-91b3e7da1018-openshift-service-ca\") pod \"30dca177-8dab-4d50-9d08-91b3e7da1018\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") "
Sep 09 21:09:07.602258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.602176    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jltlf\" (UniqueName: \"kubernetes.io/projected/30dca177-8dab-4d50-9d08-91b3e7da1018-kube-api-access-jltlf\") pod \"30dca177-8dab-4d50-9d08-91b3e7da1018\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") "
Sep 09 21:09:07.602377 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.602266    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-metrics-certs\") pod \"30dca177-8dab-4d50-9d08-91b3e7da1018\" (UID: \"30dca177-8dab-4d50-9d08-91b3e7da1018\") "
Sep 09 21:09:07.602479 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.602451    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30dca177-8dab-4d50-9d08-91b3e7da1018-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30dca177-8dab-4d50-9d08-91b3e7da1018" (UID: "30dca177-8dab-4d50-9d08-91b3e7da1018"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:07.602667 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.602645    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30dca177-8dab-4d50-9d08-91b3e7da1018-openshift-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:07.604633 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.604600    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30dca177-8dab-4d50-9d08-91b3e7da1018" (UID: "30dca177-8dab-4d50-9d08-91b3e7da1018"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:09:07.604737 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.604714    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30dca177-8dab-4d50-9d08-91b3e7da1018-kube-api-access-jltlf" (OuterVolumeSpecName: "kube-api-access-jltlf") pod "30dca177-8dab-4d50-9d08-91b3e7da1018" (UID: "30dca177-8dab-4d50-9d08-91b3e7da1018"). InnerVolumeSpecName "kube-api-access-jltlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:07.604822 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.604803    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "30dca177-8dab-4d50-9d08-91b3e7da1018" (UID: "30dca177-8dab-4d50-9d08-91b3e7da1018"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:09:07.703693 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.703659    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-metrics-certs\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:07.703693 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.703695    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30dca177-8dab-4d50-9d08-91b3e7da1018-maas-api-tls\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:07.703880 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.703710    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jltlf\" (UniqueName: \"kubernetes.io/projected/30dca177-8dab-4d50-9d08-91b3e7da1018-kube-api-access-jltlf\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:07.735989 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:07.735956    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:08.240580 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.240544    2571 generic.go:356] "Generic (PLEG): container finished" podID="30dca177-8dab-4d50-9d08-91b3e7da1018" containerID="0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933" exitCode=0
Sep 09 21:09:08.241001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.240592    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"
Sep 09 21:09:08.241001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.240624    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp" event={"ID":"30dca177-8dab-4d50-9d08-91b3e7da1018","Type":"ContainerDied","Data":"0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933"}
Sep 09 21:09:08.241001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.240675    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp" event={"ID":"30dca177-8dab-4d50-9d08-91b3e7da1018","Type":"ContainerDied","Data":"39261b79be0e04f166aa5e4f0ffbdf7914117884183dffa02883f70edb52ad08"}
Sep 09 21:09:08.241001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.240705    2571 scope.go:117] "RemoveContainer" containerID="0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933"
Sep 09 21:09:08.251517 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.251496    2571 scope.go:117] "RemoveContainer" containerID="0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933"
Sep 09 21:09:08.251844 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:09:08.251821    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933\": container with ID starting with 0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933 not found: ID does not exist" containerID="0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933"
Sep 09 21:09:08.251965 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.251855    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933"} err="failed to get container status \"0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933\": rpc error: code = NotFound desc = could not find container \"0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933\": container with ID starting with 0623b76157cac687c8524a76d271570bbdcacc87b30862a614a08d3c83b79933 not found: ID does not exist"
Sep 09 21:09:08.262526 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.262493    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"]
Sep 09 21:09:08.267043 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.267013    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9a54ea-7568cfd6b9-d82rp"]
Sep 09 21:09:08.567069 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:08.566985    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30dca177-8dab-4d50-9d08-91b3e7da1018" path="/var/lib/kubelet/pods/30dca177-8dab-4d50-9d08-91b3e7da1018/volumes"
Sep 09 21:09:13.722658 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:13.722627    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:13.823231 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:13.823190    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:14.043571 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.043479    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:14.386367 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.386268    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"]
Sep 09 21:09:14.386841 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.386821    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30dca177-8dab-4d50-9d08-91b3e7da1018" containerName="maas-api"
Sep 09 21:09:14.386942 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.386844    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="30dca177-8dab-4d50-9d08-91b3e7da1018" containerName="maas-api"
Sep 09 21:09:14.387003 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.386948    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="30dca177-8dab-4d50-9d08-91b3e7da1018" containerName="maas-api"
Sep 09 21:09:14.391669 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.391646    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"
Sep 09 21:09:14.394281 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.394244    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:09:14.397101 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.397057    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"]
Sep 09 21:09:14.467754 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.467718    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585cb470-9f50-4031-bd9b-7cada6c9b4b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn\" (UID: \"585cb470-9f50-4031-bd9b-7cada6c9b4b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"
Sep 09 21:09:14.533595 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.533562    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:14.568879 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.568842    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585cb470-9f50-4031-bd9b-7cada6c9b4b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn\" (UID: \"585cb470-9f50-4031-bd9b-7cada6c9b4b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"
Sep 09 21:09:14.569459 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.569439    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585cb470-9f50-4031-bd9b-7cada6c9b4b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn\" (UID: \"585cb470-9f50-4031-bd9b-7cada6c9b4b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"
Sep 09 21:09:14.705212 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.705180    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"
Sep 09 21:09:14.985161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:14.985131    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"]
Sep 09 21:09:14.986971 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:09:14.986948    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585cb470_9f50_4031_bd9b_7cada6c9b4b7.slice/crio-10cce1f700f9f30e21e526e328efc3dbb91468cd0ebf785a1ca9d193a8dd9fce WatchSource:0}: Error finding container 10cce1f700f9f30e21e526e328efc3dbb91468cd0ebf785a1ca9d193a8dd9fce: Status 404 returned error can't find the container with id 10cce1f700f9f30e21e526e328efc3dbb91468cd0ebf785a1ca9d193a8dd9fce
Sep 09 21:09:15.293699 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:15.293659    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn" event={"ID":"585cb470-9f50-4031-bd9b-7cada6c9b4b7","Type":"ContainerStarted","Data":"4bfff0eb14167526fdcdfd83c1e1d5d06f847a5d510895b640d1a868c9679077"}
Sep 09 21:09:15.293883 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:15.293753    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn" event={"ID":"585cb470-9f50-4031-bd9b-7cada6c9b4b7","Type":"ContainerStarted","Data":"10cce1f700f9f30e21e526e328efc3dbb91468cd0ebf785a1ca9d193a8dd9fce"}
Sep 09 21:09:15.332099 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:15.311792    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn" podStartSLOduration=1.311774163 podStartE2EDuration="1.311774163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:09:15.308847283 +0000 UTC m=+1249.279414000" watchObservedRunningTime="2026-09-09 21:09:15.311774163 +0000 UTC m=+1249.282340881"
Sep 09 21:09:16.302909 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:16.302873    2571 generic.go:356] "Generic (PLEG): container finished" podID="585cb470-9f50-4031-bd9b-7cada6c9b4b7" containerID="4bfff0eb14167526fdcdfd83c1e1d5d06f847a5d510895b640d1a868c9679077" exitCode=0
Sep 09 21:09:16.303304 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:16.302964    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn" event={"ID":"585cb470-9f50-4031-bd9b-7cada6c9b4b7","Type":"ContainerDied","Data":"4bfff0eb14167526fdcdfd83c1e1d5d06f847a5d510895b640d1a868c9679077"}
Sep 09 21:09:17.456232 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:17.456205    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"
Sep 09 21:09:17.599032 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:17.598902    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585cb470-9f50-4031-bd9b-7cada6c9b4b7-maas-api-service-ca\") pod \"585cb470-9f50-4031-bd9b-7cada6c9b4b7\" (UID: \"585cb470-9f50-4031-bd9b-7cada6c9b4b7\") "
Sep 09 21:09:17.599314 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:17.599289    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585cb470-9f50-4031-bd9b-7cada6c9b4b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "585cb470-9f50-4031-bd9b-7cada6c9b4b7" (UID: "585cb470-9f50-4031-bd9b-7cada6c9b4b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:17.700275 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:17.700232    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/585cb470-9f50-4031-bd9b-7cada6c9b4b7-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:18.243276 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:18.243246    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:18.243514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:18.243500    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:18.316598 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:18.316565    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn" event={"ID":"585cb470-9f50-4031-bd9b-7cada6c9b4b7","Type":"ContainerDied","Data":"10cce1f700f9f30e21e526e328efc3dbb91468cd0ebf785a1ca9d193a8dd9fce"}
Sep 09 21:09:18.316758 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:18.316621    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10cce1f700f9f30e21e526e328efc3dbb91468cd0ebf785a1ca9d193a8dd9fce"
Sep 09 21:09:18.316758 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:18.316582    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"
Sep 09 21:09:25.019048 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.019006    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:25.042675 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.042642    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"]
Sep 09 21:09:25.043962 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.043921    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs" podUID="3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1" containerName="payload-processing" containerID="cri-o://297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60" gracePeriod=30
Sep 09 21:09:25.047060 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.047030    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"]
Sep 09 21:09:25.047285 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.047255    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj" podUID="e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca" containerName="payload-pre-processing" containerID="cri-o://bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519" gracePeriod=30
Sep 09 21:09:25.089293 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.089256    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:25.159132 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.159070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:25.218003 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.217971    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"]
Sep 09 21:09:25.223976 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.223946    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-98dad6-2t7kn"]
Sep 09 21:09:25.240183 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.240137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:25.334435 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.334403    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:09:25.337233 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.337215    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:09:25.362124 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.362064    2571 generic.go:356] "Generic (PLEG): container finished" podID="e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca" containerID="bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519" exitCode=0
Sep 09 21:09:25.362124 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.362126    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"
Sep 09 21:09:25.362377 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.362137    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj" event={"ID":"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca","Type":"ContainerDied","Data":"bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519"}
Sep 09 21:09:25.362377 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.362199    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj" event={"ID":"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca","Type":"ContainerDied","Data":"a3b932bdb2744308099eb7d47916158572abeb018950dc62e14bfad132a62708"}
Sep 09 21:09:25.362377 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.362229    2571 scope.go:117] "RemoveContainer" containerID="bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519"
Sep 09 21:09:25.363681 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.363660    2571 generic.go:356] "Generic (PLEG): container finished" podID="3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1" containerID="297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60" exitCode=0
Sep 09 21:09:25.363784 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.363712    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"
Sep 09 21:09:25.363784 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.363750    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs" event={"ID":"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1","Type":"ContainerDied","Data":"297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60"}
Sep 09 21:09:25.363867 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.363800    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs" event={"ID":"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1","Type":"ContainerDied","Data":"6e8ce08af085e8fee05e0adecf435522dc38e54a4829b25a9ed7a7752183ef6c"}
Sep 09 21:09:25.374421 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.374386    2571 scope.go:117] "RemoveContainer" containerID="bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519"
Sep 09 21:09:25.374836 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:09:25.374802    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519\": container with ID starting with bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519 not found: ID does not exist" containerID="bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519"
Sep 09 21:09:25.374961 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.374847    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519"} err="failed to get container status \"bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519\": rpc error: code = NotFound desc = could not find container \"bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519\": container with ID starting with bee6a01b121a46fa39c4dd5a80afd01fec9f22d61dddbfa917b5e565b74c9519 not found: ID does not exist"
Sep 09 21:09:25.374961 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.374872    2571 scope.go:117] "RemoveContainer" containerID="297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60"
Sep 09 21:09:25.386577 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.386551    2571 scope.go:117] "RemoveContainer" containerID="297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60"
Sep 09 21:09:25.386918 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:09:25.386893    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60\": container with ID starting with 297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60 not found: ID does not exist" containerID="297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60"
Sep 09 21:09:25.386986 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.386928    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60"} err="failed to get container status \"297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60\": rpc error: code = NotFound desc = could not find container \"297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60\": container with ID starting with 297da62d37bf7d3cc7341daf3040d823d5133f11454dc9934f4b991b55fe1a60 not found: ID does not exist"
Sep 09 21:09:25.476189 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.476153    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvngp\" (UniqueName: \"kubernetes.io/projected/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-kube-api-access-gvngp\") pod \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\" (UID: \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\") "
Sep 09 21:09:25.476380 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.476223    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-plugins-config-volume\") pod \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\" (UID: \"3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1\") "
Sep 09 21:09:25.476380 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.476261    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-plugins-config-volume\") pod \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\" (UID: \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\") "
Sep 09 21:09:25.476380 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.476281    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv6wn\" (UniqueName: \"kubernetes.io/projected/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-kube-api-access-qv6wn\") pod \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\" (UID: \"e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca\") "
Sep 09 21:09:25.476663 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.476638    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca" (UID: "e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:25.476724 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.476638    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1" (UID: "3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:25.478585 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.478567    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-kube-api-access-qv6wn" (OuterVolumeSpecName: "kube-api-access-qv6wn") pod "e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca" (UID: "e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca"). InnerVolumeSpecName "kube-api-access-qv6wn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:25.478654 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.478635    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-kube-api-access-gvngp" (OuterVolumeSpecName: "kube-api-access-gvngp") pod "3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1" (UID: "3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1"). InnerVolumeSpecName "kube-api-access-gvngp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:25.577225 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.577138    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:25.577225 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.577166    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv6wn\" (UniqueName: \"kubernetes.io/projected/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca-kube-api-access-qv6wn\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:25.577225 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.577175    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvngp\" (UniqueName: \"kubernetes.io/projected/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-kube-api-access-gvngp\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:25.577225 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.577185    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:25.654266 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.654229    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:25.692860 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.692827    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"]
Sep 09 21:09:25.697247 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.697217    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-98dad6-5764858bdc-dxsjs"]
Sep 09 21:09:25.708198 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.708169    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"]
Sep 09 21:09:25.711846 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:25.711820    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-98dad6-67b55d9bdb-qttlj"]
Sep 09 21:09:26.573806 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:26.573776    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1" path="/var/lib/kubelet/pods/3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1/volumes"
Sep 09 21:09:26.574196 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:26.574120    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585cb470-9f50-4031-bd9b-7cada6c9b4b7" path="/var/lib/kubelet/pods/585cb470-9f50-4031-bd9b-7cada6c9b4b7/volumes"
Sep 09 21:09:26.574470 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:26.574456    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca" path="/var/lib/kubelet/pods/e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca/volumes"
Sep 09 21:09:26.791369 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:26.791332    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:29.707015 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:29.706977    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:30.468475 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:30.468436    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:30.566638 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:30.566603    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:30.963002 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:30.962972    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:32.479114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:32.479065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:32.576724 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:32.576690    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:32.694348 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:32.694312    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:33.262835 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:33.262796    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:33.568815 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:33.568719    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:33.839303 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:33.839186    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:36.548094 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:36.548025    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:36.697108 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:36.697054    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:37.862608 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:37.862571    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:37.962808 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:37.962768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:38.059447 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.059413    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:38.371042 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.371007    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:38.470677 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.470645    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:38.569226 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569189    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"]
Sep 09 21:09:38.569647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569633    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca" containerName="payload-pre-processing"
Sep 09 21:09:38.569698 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569650    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca" containerName="payload-pre-processing"
Sep 09 21:09:38.569698 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569659    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585cb470-9f50-4031-bd9b-7cada6c9b4b7" containerName="revoke-keys"
Sep 09 21:09:38.569698 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569665    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="585cb470-9f50-4031-bd9b-7cada6c9b4b7" containerName="revoke-keys"
Sep 09 21:09:38.569698 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569675    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1" containerName="payload-processing"
Sep 09 21:09:38.569698 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569680    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1" containerName="payload-processing"
Sep 09 21:09:38.569861 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569755    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="585cb470-9f50-4031-bd9b-7cada6c9b4b7" containerName="revoke-keys"
Sep 09 21:09:38.569861 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569767    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e8968d1-d9d8-4f94-b7a9-f3842cd3f6f1" containerName="payload-processing"
Sep 09 21:09:38.569861 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.569777    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9e42b09-9cd1-4f27-85c9-3c5b9ef219ca" containerName="payload-pre-processing"
Sep 09 21:09:38.574746 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.574728    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:38.578093 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.578055    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-3d3cac-dockercfg-jnkn5\""
Sep 09 21:09:38.578506 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.578489    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-3d3cac\""
Sep 09 21:09:38.589781 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.589756    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"]
Sep 09 21:09:38.595597 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.595570    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a6f0b37-4214-499a-aa6b-54859916bca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c\" (UID: \"6a6f0b37-4214-499a-aa6b-54859916bca9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:38.595755 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.595631    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4x62\" (UniqueName: \"kubernetes.io/projected/6a6f0b37-4214-499a-aa6b-54859916bca9-kube-api-access-b4x62\") pod \"payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c\" (UID: \"6a6f0b37-4214-499a-aa6b-54859916bca9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:38.696215 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.696183    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4x62\" (UniqueName: \"kubernetes.io/projected/6a6f0b37-4214-499a-aa6b-54859916bca9-kube-api-access-b4x62\") pod \"payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c\" (UID: \"6a6f0b37-4214-499a-aa6b-54859916bca9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:38.696382 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.696290    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a6f0b37-4214-499a-aa6b-54859916bca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c\" (UID: \"6a6f0b37-4214-499a-aa6b-54859916bca9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:38.696952 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.696932    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a6f0b37-4214-499a-aa6b-54859916bca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c\" (UID: \"6a6f0b37-4214-499a-aa6b-54859916bca9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:38.706307 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.706283    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4x62\" (UniqueName: \"kubernetes.io/projected/6a6f0b37-4214-499a-aa6b-54859916bca9-kube-api-access-b4x62\") pod \"payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c\" (UID: \"6a6f0b37-4214-499a-aa6b-54859916bca9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:38.813827 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.813795    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:38.885665 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.885630    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:38.905969 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:38.905924    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:39.158925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.158837    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"]
Sep 09 21:09:39.164040 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.164019    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"
Sep 09 21:09:39.164472 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.164446    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"]
Sep 09 21:09:39.166292 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.166264    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:09:39.166536 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:09:39.166509    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a6f0b37_4214_499a_aa6b_54859916bca9.slice/crio-6841b3f2dca2254e49e4ead0dbeba2ac5981667d520bd5f435a54500e0425cef WatchSource:0}: Error finding container 6841b3f2dca2254e49e4ead0dbeba2ac5981667d520bd5f435a54500e0425cef: Status 404 returned error can't find the container with id 6841b3f2dca2254e49e4ead0dbeba2ac5981667d520bd5f435a54500e0425cef
Sep 09 21:09:39.171438 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.171397    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"]
Sep 09 21:09:39.202489 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.202459    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97384a7e-a457-47e3-9d3a-e374e3a4518b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk\" (UID: \"97384a7e-a457-47e3-9d3a-e374e3a4518b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"
Sep 09 21:09:39.303546 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.303512    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97384a7e-a457-47e3-9d3a-e374e3a4518b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk\" (UID: \"97384a7e-a457-47e3-9d3a-e374e3a4518b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"
Sep 09 21:09:39.304151 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.304126    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97384a7e-a457-47e3-9d3a-e374e3a4518b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk\" (UID: \"97384a7e-a457-47e3-9d3a-e374e3a4518b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"
Sep 09 21:09:39.464039 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.464002    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c" event={"ID":"6a6f0b37-4214-499a-aa6b-54859916bca9","Type":"ContainerStarted","Data":"878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5"}
Sep 09 21:09:39.464270 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.464065    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:39.464270 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.464104    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c" event={"ID":"6a6f0b37-4214-499a-aa6b-54859916bca9","Type":"ContainerStarted","Data":"6841b3f2dca2254e49e4ead0dbeba2ac5981667d520bd5f435a54500e0425cef"}
Sep 09 21:09:39.490095 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.490028    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c" podStartSLOduration=1.49001117 podStartE2EDuration="1.49001117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:09:39.488124189 +0000 UTC m=+1273.458690906" watchObservedRunningTime="2026-09-09 21:09:39.49001117 +0000 UTC m=+1273.460577887"
Sep 09 21:09:39.503707 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.503680    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"
Sep 09 21:09:39.534330 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.534300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:39.644449 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.644420    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:39.779314 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:39.779030    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"]
Sep 09 21:09:39.782924 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:09:39.782897    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97384a7e_a457_47e3_9d3a_e374e3a4518b.slice/crio-a20ffb4403e2fc5b38563a19fe3635e6cf3e2db3aa59c62e9f1cd27eac3d4433 WatchSource:0}: Error finding container a20ffb4403e2fc5b38563a19fe3635e6cf3e2db3aa59c62e9f1cd27eac3d4433: Status 404 returned error can't find the container with id a20ffb4403e2fc5b38563a19fe3635e6cf3e2db3aa59c62e9f1cd27eac3d4433
Sep 09 21:09:40.473820 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:40.473702    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk" event={"ID":"97384a7e-a457-47e3-9d3a-e374e3a4518b","Type":"ContainerStarted","Data":"1f0de9488ff621628206ff2e498918bb2e6497daab25a779423c2a51a083c69f"}
Sep 09 21:09:40.473820 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:40.473758    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk" event={"ID":"97384a7e-a457-47e3-9d3a-e374e3a4518b","Type":"ContainerStarted","Data":"a20ffb4403e2fc5b38563a19fe3635e6cf3e2db3aa59c62e9f1cd27eac3d4433"}
Sep 09 21:09:40.549024 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:40.548994    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:41.481613 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:41.481578    2571 generic.go:356] "Generic (PLEG): container finished" podID="97384a7e-a457-47e3-9d3a-e374e3a4518b" containerID="1f0de9488ff621628206ff2e498918bb2e6497daab25a779423c2a51a083c69f" exitCode=0
Sep 09 21:09:41.482041 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:41.481666    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk" event={"ID":"97384a7e-a457-47e3-9d3a-e374e3a4518b","Type":"ContainerDied","Data":"1f0de9488ff621628206ff2e498918bb2e6497daab25a779423c2a51a083c69f"}
Sep 09 21:09:41.624655 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:41.624628    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"
Sep 09 21:09:41.728272 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:41.728241    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97384a7e-a457-47e3-9d3a-e374e3a4518b-maas-api-service-ca\") pod \"97384a7e-a457-47e3-9d3a-e374e3a4518b\" (UID: \"97384a7e-a457-47e3-9d3a-e374e3a4518b\") "
Sep 09 21:09:41.728575 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:41.728556    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97384a7e-a457-47e3-9d3a-e374e3a4518b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97384a7e-a457-47e3-9d3a-e374e3a4518b" (UID: "97384a7e-a457-47e3-9d3a-e374e3a4518b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:41.829491 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:41.829401    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97384a7e-a457-47e3-9d3a-e374e3a4518b-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:42.488981 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:42.488945    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk" event={"ID":"97384a7e-a457-47e3-9d3a-e374e3a4518b","Type":"ContainerDied","Data":"a20ffb4403e2fc5b38563a19fe3635e6cf3e2db3aa59c62e9f1cd27eac3d4433"}
Sep 09 21:09:42.489458 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:42.489003    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a20ffb4403e2fc5b38563a19fe3635e6cf3e2db3aa59c62e9f1cd27eac3d4433"
Sep 09 21:09:42.489458 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:42.488954    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"
Sep 09 21:09:49.364750 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:49.364710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:50.061409 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.061364    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:50.085324 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.085294    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"]
Sep 09 21:09:50.085532 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.085508    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv" podUID="9522241c-da90-45d2-a198-5618a7d8e1ad" containerName="payload-processing" containerID="cri-o://10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1" gracePeriod=30
Sep 09 21:09:50.098421 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.094992    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"]
Sep 09 21:09:50.098421 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.095317    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz" podUID="82521c56-a08f-40e5-99e0-4915cf317d5b" containerName="payload-pre-processing" containerID="cri-o://ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7" gracePeriod=30
Sep 09 21:09:50.160294 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.160259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:50.261034 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.260999    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:50.287743 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.287674    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"]
Sep 09 21:09:50.290728 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.290680    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-57783aa9-j7fnk"]
Sep 09 21:09:50.357975 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.357946    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:50.363502 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.363474    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:50.390022 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.389992    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:50.416015 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.415987    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9522241c-da90-45d2-a198-5618a7d8e1ad-plugins-config-volume\") pod \"9522241c-da90-45d2-a198-5618a7d8e1ad\" (UID: \"9522241c-da90-45d2-a198-5618a7d8e1ad\") "
Sep 09 21:09:50.416192 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.416041    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txhwl\" (UniqueName: \"kubernetes.io/projected/9522241c-da90-45d2-a198-5618a7d8e1ad-kube-api-access-txhwl\") pod \"9522241c-da90-45d2-a198-5618a7d8e1ad\" (UID: \"9522241c-da90-45d2-a198-5618a7d8e1ad\") "
Sep 09 21:09:50.416458 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.416428    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9522241c-da90-45d2-a198-5618a7d8e1ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9522241c-da90-45d2-a198-5618a7d8e1ad" (UID: "9522241c-da90-45d2-a198-5618a7d8e1ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:50.418598 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.418576    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9522241c-da90-45d2-a198-5618a7d8e1ad-kube-api-access-txhwl" (OuterVolumeSpecName: "kube-api-access-txhwl") pod "9522241c-da90-45d2-a198-5618a7d8e1ad" (UID: "9522241c-da90-45d2-a198-5618a7d8e1ad"). InnerVolumeSpecName "kube-api-access-txhwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:50.476274 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.476239    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:09:50.517220 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.517190    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpcqh\" (UniqueName: \"kubernetes.io/projected/82521c56-a08f-40e5-99e0-4915cf317d5b-kube-api-access-cpcqh\") pod \"82521c56-a08f-40e5-99e0-4915cf317d5b\" (UID: \"82521c56-a08f-40e5-99e0-4915cf317d5b\") "
Sep 09 21:09:50.517426 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.517248    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82521c56-a08f-40e5-99e0-4915cf317d5b-plugins-config-volume\") pod \"82521c56-a08f-40e5-99e0-4915cf317d5b\" (UID: \"82521c56-a08f-40e5-99e0-4915cf317d5b\") "
Sep 09 21:09:50.517976 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.517649    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82521c56-a08f-40e5-99e0-4915cf317d5b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82521c56-a08f-40e5-99e0-4915cf317d5b" (UID: "82521c56-a08f-40e5-99e0-4915cf317d5b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:09:50.517976 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.517720    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txhwl\" (UniqueName: \"kubernetes.io/projected/9522241c-da90-45d2-a198-5618a7d8e1ad-kube-api-access-txhwl\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:50.517976 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.517741    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9522241c-da90-45d2-a198-5618a7d8e1ad-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:50.519838 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.519817    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82521c56-a08f-40e5-99e0-4915cf317d5b-kube-api-access-cpcqh" (OuterVolumeSpecName: "kube-api-access-cpcqh") pod "82521c56-a08f-40e5-99e0-4915cf317d5b" (UID: "82521c56-a08f-40e5-99e0-4915cf317d5b"). InnerVolumeSpecName "kube-api-access-cpcqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:09:50.538365 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.538332    2571 generic.go:356] "Generic (PLEG): container finished" podID="82521c56-a08f-40e5-99e0-4915cf317d5b" containerID="ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7" exitCode=0
Sep 09 21:09:50.538524 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.538383    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"
Sep 09 21:09:50.538524 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.538446    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz" event={"ID":"82521c56-a08f-40e5-99e0-4915cf317d5b","Type":"ContainerDied","Data":"ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7"}
Sep 09 21:09:50.538524 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.538505    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz" event={"ID":"82521c56-a08f-40e5-99e0-4915cf317d5b","Type":"ContainerDied","Data":"6e5480b7e3b52d47759c3b43f27d6998e371c9fa8d8a9011bb63b2cf1e06c5ee"}
Sep 09 21:09:50.538682 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.538535    2571 scope.go:117] "RemoveContainer" containerID="ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7"
Sep 09 21:09:50.541019 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.540991    2571 generic.go:356] "Generic (PLEG): container finished" podID="9522241c-da90-45d2-a198-5618a7d8e1ad" containerID="10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1" exitCode=0
Sep 09 21:09:50.541161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.541035    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv" event={"ID":"9522241c-da90-45d2-a198-5618a7d8e1ad","Type":"ContainerDied","Data":"10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1"}
Sep 09 21:09:50.541161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.541053    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"
Sep 09 21:09:50.541161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.541109    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv" event={"ID":"9522241c-da90-45d2-a198-5618a7d8e1ad","Type":"ContainerDied","Data":"0fbe6c73fde67299831b62293ec2b0a95a44d3f0ce8347df49a03e1de6020de9"}
Sep 09 21:09:50.549683 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.549629    2571 scope.go:117] "RemoveContainer" containerID="ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7"
Sep 09 21:09:50.550006 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:09:50.549984    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7\": container with ID starting with ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7 not found: ID does not exist" containerID="ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7"
Sep 09 21:09:50.550140 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.550017    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7"} err="failed to get container status \"ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7\": rpc error: code = NotFound desc = could not find container \"ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7\": container with ID starting with ff869fdd921916ff94ceb327c5f6f2caaec0c7d37e31c36e4240e0220b0257d7 not found: ID does not exist"
Sep 09 21:09:50.550140 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.550041    2571 scope.go:117] "RemoveContainer" containerID="10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1"
Sep 09 21:09:50.559062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.559047    2571 scope.go:117] "RemoveContainer" containerID="10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1"
Sep 09 21:09:50.559392 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:09:50.559369    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1\": container with ID starting with 10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1 not found: ID does not exist" containerID="10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1"
Sep 09 21:09:50.559469 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.559403    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1"} err="failed to get container status \"10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1\": rpc error: code = NotFound desc = could not find container \"10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1\": container with ID starting with 10c0c21df29d349112eafa834211bbaa8a40135e842da60fbe38a9956d7437a1 not found: ID does not exist"
Sep 09 21:09:50.566633 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.566570    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97384a7e-a457-47e3-9d3a-e374e3a4518b" path="/var/lib/kubelet/pods/97384a7e-a457-47e3-9d3a-e374e3a4518b/volumes"
Sep 09 21:09:50.571126 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.571104    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"]
Sep 09 21:09:50.573574 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.573556    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-57783aa9-7cb9ccd969-p2tbv"]
Sep 09 21:09:50.585783 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.585746    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"]
Sep 09 21:09:50.590201 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.590181    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-57783aa9-59df4d6594-qv7lz"]
Sep 09 21:09:50.618297 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.618256    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82521c56-a08f-40e5-99e0-4915cf317d5b-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:50.618297 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.618291    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpcqh\" (UniqueName: \"kubernetes.io/projected/82521c56-a08f-40e5-99e0-4915cf317d5b-kube-api-access-cpcqh\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:09:50.691030 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:50.690983    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:52.566601 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:52.566569    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82521c56-a08f-40e5-99e0-4915cf317d5b" path="/var/lib/kubelet/pods/82521c56-a08f-40e5-99e0-4915cf317d5b/volumes"
Sep 09 21:09:52.566971 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:52.566929    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9522241c-da90-45d2-a198-5618a7d8e1ad" path="/var/lib/kubelet/pods/9522241c-da90-45d2-a198-5618a7d8e1ad/volumes"
Sep 09 21:09:56.497049 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:56.497017    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:56.565010 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:56.564974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:56.753280 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:56.753193    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:57.132667 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:57.132561    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:57.298111 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:57.298060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:57.445565 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:57.445533    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:09:58.230451 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:09:58.230402    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:01.228519 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:01.228482    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:03.535640 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:03.535601    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:03.629767 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:03.629730    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:03.754300 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:03.754261    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:06.608392 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:06.608361    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:06.930144 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:06.930039    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:07.239698 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:07.239666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:07.708494 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:07.708463    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:09.881158 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:09.881124    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:09.953173 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:09.953134    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:10.058911 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:10.058864    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:10.150643 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:10.150552    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:10.795011 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:10.794972    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:11.098630 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:11.098537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:11.354161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:11.354045    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:11.454497 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:11.454458    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:11.561940 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:11.561911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:11.653939 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:11.653850    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:11.920491 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:11.920401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:12.007673 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.007623    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:12.063550 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.063512    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"]
Sep 09 21:10:12.063968 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.063956    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9522241c-da90-45d2-a198-5618a7d8e1ad" containerName="payload-processing"
Sep 09 21:10:12.064016 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.063969    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9522241c-da90-45d2-a198-5618a7d8e1ad" containerName="payload-processing"
Sep 09 21:10:12.064016 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.063984    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82521c56-a08f-40e5-99e0-4915cf317d5b" containerName="payload-pre-processing"
Sep 09 21:10:12.064016 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.063990    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="82521c56-a08f-40e5-99e0-4915cf317d5b" containerName="payload-pre-processing"
Sep 09 21:10:12.064016 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.064000    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97384a7e-a457-47e3-9d3a-e374e3a4518b" containerName="revoke-keys"
Sep 09 21:10:12.064016 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.064005    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="97384a7e-a457-47e3-9d3a-e374e3a4518b" containerName="revoke-keys"
Sep 09 21:10:12.064232 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.064066    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="97384a7e-a457-47e3-9d3a-e374e3a4518b" containerName="revoke-keys"
Sep 09 21:10:12.064232 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.064091    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9522241c-da90-45d2-a198-5618a7d8e1ad" containerName="payload-processing"
Sep 09 21:10:12.064232 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.064101    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="82521c56-a08f-40e5-99e0-4915cf317d5b" containerName="payload-pre-processing"
Sep 09 21:10:12.067302 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.067288    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:12.069652 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.069628    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-8db5a7\""
Sep 09 21:10:12.069769 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.069703    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-8db5a7-dockercfg-kwn5f\""
Sep 09 21:10:12.078256 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.078229    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"]
Sep 09 21:10:12.107542 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.107508    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"]
Sep 09 21:10:12.111638 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.111613    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:12.125294 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.125265    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"]
Sep 09 21:10:12.137987 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.137953    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9ed0c9-46ea-4cab-8895-53ba26c45254-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5\" (UID: \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:12.138180 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.138019    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12a51d9-9437-4591-b3a3-3593ac78e5de-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c\" (UID: \"a12a51d9-9437-4591-b3a3-3593ac78e5de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:12.138180 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.138051    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4jb8\" (UniqueName: \"kubernetes.io/projected/a12a51d9-9437-4591-b3a3-3593ac78e5de-kube-api-access-k4jb8\") pod \"payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c\" (UID: \"a12a51d9-9437-4591-b3a3-3593ac78e5de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:12.138180 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.138131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llpfm\" (UniqueName: \"kubernetes.io/projected/3e9ed0c9-46ea-4cab-8895-53ba26c45254-kube-api-access-llpfm\") pod \"payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5\" (UID: \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:12.239429 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.239371    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12a51d9-9437-4591-b3a3-3593ac78e5de-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c\" (UID: \"a12a51d9-9437-4591-b3a3-3593ac78e5de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:12.239429 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.239434    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4jb8\" (UniqueName: \"kubernetes.io/projected/a12a51d9-9437-4591-b3a3-3593ac78e5de-kube-api-access-k4jb8\") pod \"payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c\" (UID: \"a12a51d9-9437-4591-b3a3-3593ac78e5de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:12.239710 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.239495    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llpfm\" (UniqueName: \"kubernetes.io/projected/3e9ed0c9-46ea-4cab-8895-53ba26c45254-kube-api-access-llpfm\") pod \"payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5\" (UID: \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:12.239710 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.239575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9ed0c9-46ea-4cab-8895-53ba26c45254-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5\" (UID: \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:12.240318 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.240293    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12a51d9-9437-4591-b3a3-3593ac78e5de-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c\" (UID: \"a12a51d9-9437-4591-b3a3-3593ac78e5de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:12.240419 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.240300    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9ed0c9-46ea-4cab-8895-53ba26c45254-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5\" (UID: \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:12.247302 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.247278    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llpfm\" (UniqueName: \"kubernetes.io/projected/3e9ed0c9-46ea-4cab-8895-53ba26c45254-kube-api-access-llpfm\") pod \"payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5\" (UID: \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:12.247415 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.247282    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4jb8\" (UniqueName: \"kubernetes.io/projected/a12a51d9-9437-4591-b3a3-3593ac78e5de-kube-api-access-k4jb8\") pod \"payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c\" (UID: \"a12a51d9-9437-4591-b3a3-3593ac78e5de\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:12.379668 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.379623    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:12.422708 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.422675    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:12.653737 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.653711    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"]
Sep 09 21:10:12.654662 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:10:12.654634    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12a51d9_9437_4591_b3a3_3593ac78e5de.slice/crio-70083ec71f36bf4bf685094c5bdab5ebf41e07058253eade6570a452a5721734 WatchSource:0}: Error finding container 70083ec71f36bf4bf685094c5bdab5ebf41e07058253eade6570a452a5721734: Status 404 returned error can't find the container with id 70083ec71f36bf4bf685094c5bdab5ebf41e07058253eade6570a452a5721734
Sep 09 21:10:12.679642 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.679617    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c" event={"ID":"a12a51d9-9437-4591-b3a3-3593ac78e5de","Type":"ContainerStarted","Data":"70083ec71f36bf4bf685094c5bdab5ebf41e07058253eade6570a452a5721734"}
Sep 09 21:10:12.709982 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:12.709952    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"]
Sep 09 21:10:12.711457 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:10:12.711431    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9ed0c9_46ea_4cab_8895_53ba26c45254.slice/crio-f42fdd43706876e2745bc89dc481611bff9b703d69095fa210fd9a3db47d2f22 WatchSource:0}: Error finding container f42fdd43706876e2745bc89dc481611bff9b703d69095fa210fd9a3db47d2f22: Status 404 returned error can't find the container with id f42fdd43706876e2745bc89dc481611bff9b703d69095fa210fd9a3db47d2f22
Sep 09 21:10:13.694881 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:13.694838    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c" event={"ID":"a12a51d9-9437-4591-b3a3-3593ac78e5de","Type":"ContainerStarted","Data":"7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb"}
Sep 09 21:10:13.695380 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:13.694936    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:13.696520 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:13.696489    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5" event={"ID":"3e9ed0c9-46ea-4cab-8895-53ba26c45254","Type":"ContainerStarted","Data":"7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c"}
Sep 09 21:10:13.696649 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:13.696542    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:13.696649 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:13.696564    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5" event={"ID":"3e9ed0c9-46ea-4cab-8895-53ba26c45254","Type":"ContainerStarted","Data":"f42fdd43706876e2745bc89dc481611bff9b703d69095fa210fd9a3db47d2f22"}
Sep 09 21:10:13.711994 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:13.711925    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c" podStartSLOduration=1.711906218 podStartE2EDuration="1.711906218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:13.710716413 +0000 UTC m=+1307.681283130" watchObservedRunningTime="2026-09-09 21:10:13.711906218 +0000 UTC m=+1307.682472936"
Sep 09 21:10:13.726208 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:13.726143    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5" podStartSLOduration=1.7261277000000002 podStartE2EDuration="1.7261277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:13.725560962 +0000 UTC m=+1307.696127679" watchObservedRunningTime="2026-09-09 21:10:13.7261277 +0000 UTC m=+1307.696694422"
Sep 09 21:10:13.924530 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:13.924494    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:14.021231 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:14.021129    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.366615 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:15.366576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.448248 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:15.448193    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.548508 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:15.548467    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.865508 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:15.865472    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:15.976743 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:15.976707    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:16.001586 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.001550    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"]
Sep 09 21:10:16.006476 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.006447    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.009367 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.009334    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-629537-dockercfg-644j9\""
Sep 09 21:10:16.009535 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.009372    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-629537\""
Sep 09 21:10:16.014307 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.014284    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"]
Sep 09 21:10:16.075796 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.075762    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c14ae5-74d2-42a3-89a1-ac7357970c12-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs\" (UID: \"65c14ae5-74d2-42a3-89a1-ac7357970c12\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.075970 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.075827    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxhj2\" (UniqueName: \"kubernetes.io/projected/65c14ae5-74d2-42a3-89a1-ac7357970c12-kube-api-access-cxhj2\") pod \"payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs\" (UID: \"65c14ae5-74d2-42a3-89a1-ac7357970c12\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.176887 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.176801    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c14ae5-74d2-42a3-89a1-ac7357970c12-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs\" (UID: \"65c14ae5-74d2-42a3-89a1-ac7357970c12\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.176887 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.176853    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxhj2\" (UniqueName: \"kubernetes.io/projected/65c14ae5-74d2-42a3-89a1-ac7357970c12-kube-api-access-cxhj2\") pod \"payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs\" (UID: \"65c14ae5-74d2-42a3-89a1-ac7357970c12\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.177543 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.177525    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c14ae5-74d2-42a3-89a1-ac7357970c12-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs\" (UID: \"65c14ae5-74d2-42a3-89a1-ac7357970c12\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.187669 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.187644    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxhj2\" (UniqueName: \"kubernetes.io/projected/65c14ae5-74d2-42a3-89a1-ac7357970c12-kube-api-access-cxhj2\") pod \"payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs\" (UID: \"65c14ae5-74d2-42a3-89a1-ac7357970c12\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.319567 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.319538    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.598288 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.598263    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"]
Sep 09 21:10:16.599125 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:10:16.599095    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c14ae5_74d2_42a3_89a1_ac7357970c12.slice/crio-94a2d68523f30cdd6816d7bd8f0e8d0f8135ff1ee6b47b34512d3608cee90fe3 WatchSource:0}: Error finding container 94a2d68523f30cdd6816d7bd8f0e8d0f8135ff1ee6b47b34512d3608cee90fe3: Status 404 returned error can't find the container with id 94a2d68523f30cdd6816d7bd8f0e8d0f8135ff1ee6b47b34512d3608cee90fe3
Sep 09 21:10:16.720386 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.720347    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs" event={"ID":"65c14ae5-74d2-42a3-89a1-ac7357970c12","Type":"ContainerStarted","Data":"ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e"}
Sep 09 21:10:16.720575 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.720408    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs" event={"ID":"65c14ae5-74d2-42a3-89a1-ac7357970c12","Type":"ContainerStarted","Data":"94a2d68523f30cdd6816d7bd8f0e8d0f8135ff1ee6b47b34512d3608cee90fe3"}
Sep 09 21:10:16.720575 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.720448    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:16.764002 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:16.763947    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs" podStartSLOduration=1.763930211 podStartE2EDuration="1.763930211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:16.762891509 +0000 UTC m=+1310.733458226" watchObservedRunningTime="2026-09-09 21:10:16.763930211 +0000 UTC m=+1310.734496927"
Sep 09 21:10:24.704697 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:24.704666    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:10:24.705277 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:24.704880    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:10:27.729844 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:27.729807    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:10:32.340912 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:32.340880    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:34.964947 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:34.964912    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:37.575624 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:37.575587    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:38.926344 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:38.926307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:39.274999 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:39.274962    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:39.574833 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:39.574737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:40.387994 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:40.387937    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:43.098116 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:43.098038    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:43.187234 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:43.187195    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:44.243758 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.243725    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:44.433786 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.433755    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:44.540143 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.540051    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:44.840668 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.840584    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:44.939704 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.939667    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:44.945561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.945536    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"]
Sep 09 21:10:44.950611 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.950595    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:44.953019 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.952997    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-459158\""
Sep 09 21:10:44.953206 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.953188    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-459158-dockercfg-n6z7h\""
Sep 09 21:10:44.964423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:44.964402    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"]
Sep 09 21:10:45.068791 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.068757    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx\" (UID: \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:45.069004 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.068824    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhkqc\" (UniqueName: \"kubernetes.io/projected/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-kube-api-access-rhkqc\") pod \"payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx\" (UID: \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:45.169874 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.169782    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx\" (UID: \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:45.169874 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.169832    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhkqc\" (UniqueName: \"kubernetes.io/projected/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-kube-api-access-rhkqc\") pod \"payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx\" (UID: \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:45.170577 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.170559    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx\" (UID: \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:45.177617 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.177594    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhkqc\" (UniqueName: \"kubernetes.io/projected/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-kube-api-access-rhkqc\") pod \"payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx\" (UID: \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:45.260682 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.260649    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:45.537450 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.537420    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"]
Sep 09 21:10:45.537588 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:10:45.537475    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3e0aa2_d88d_466f_b48f_5e6b3ff1affe.slice/crio-01d4ee2c1feb3633850a461f864c21cc22df3e8d1a9e8c18f5e70761f7c9777e WatchSource:0}: Error finding container 01d4ee2c1feb3633850a461f864c21cc22df3e8d1a9e8c18f5e70761f7c9777e: Status 404 returned error can't find the container with id 01d4ee2c1feb3633850a461f864c21cc22df3e8d1a9e8c18f5e70761f7c9777e
Sep 09 21:10:45.901605 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.901518    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx" event={"ID":"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe","Type":"ContainerStarted","Data":"c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51"}
Sep 09 21:10:45.901605 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.901570    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx" event={"ID":"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe","Type":"ContainerStarted","Data":"01d4ee2c1feb3633850a461f864c21cc22df3e8d1a9e8c18f5e70761f7c9777e"}
Sep 09 21:10:45.901879 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.901663    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:10:45.918965 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:45.918912    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx" podStartSLOduration=1.918889066 podStartE2EDuration="1.918889066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:45.917483104 +0000 UTC m=+1339.888049827" watchObservedRunningTime="2026-09-09 21:10:45.918889066 +0000 UTC m=+1339.889455784"
Sep 09 21:10:50.505389 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.505290    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"]
Sep 09 21:10:50.509572 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.509551    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.512754 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.512702    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3d3cac\"/\"default-dockercfg-vb4x7\""
Sep 09 21:10:50.512754 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.512719    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3d3cac\"/\"kube-root-ca.crt\""
Sep 09 21:10:50.512976 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.512783    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3d3cac\"/\"openshift-service-ca.crt\""
Sep 09 21:10:50.512976 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.512799    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3d3cac\"/\"auth-test-model-3d3cac-kserve-self-signed-certs\""
Sep 09 21:10:50.519954 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.519927    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntm9c\" (UniqueName: \"kubernetes.io/projected/5552cc45-a271-440e-b925-9c4ad819af36-kube-api-access-ntm9c\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.520130 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.519985    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-home\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.520130 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.520022    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5552cc45-a271-440e-b925-9c4ad819af36-tls-certs\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.520257 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.520137    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-tmp-dir\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.520308 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.520265    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-model-cache\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.520308 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.520296    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-dshm\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.522956 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.522933    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"]
Sep 09 21:10:50.541049 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.541015    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:10:50.620724 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.620684    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-home\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.620931 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.620742    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5552cc45-a271-440e-b925-9c4ad819af36-tls-certs\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.620931 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.620799    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-tmp-dir\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.620931 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.620868    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-model-cache\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.620931 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.620885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-dshm\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.620931 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.620911    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntm9c\" (UniqueName: \"kubernetes.io/projected/5552cc45-a271-440e-b925-9c4ad819af36-kube-api-access-ntm9c\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.621260 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.621174    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-home\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.621307 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.621258    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-tmp-dir\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.621390 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.621372    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-model-cache\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.623297 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.623269    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-dshm\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.623448 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.623432    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5552cc45-a271-440e-b925-9c4ad819af36-tls-certs\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.629849 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.629826    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntm9c\" (UniqueName: \"kubernetes.io/projected/5552cc45-a271-440e-b925-9c4ad819af36-kube-api-access-ntm9c\") pod \"auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") " pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:50.820555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:50.820454    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:10:51.102706 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:51.102676    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"]
Sep 09 21:10:51.103775 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:10:51.103752    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5552cc45_a271_440e_b925_9c4ad819af36.slice/crio-b7031cc027c13cd7a843ae57dab930f18b02c7a300228cc8ce7c9502f2cc05f6 WatchSource:0}: Error finding container b7031cc027c13cd7a843ae57dab930f18b02c7a300228cc8ce7c9502f2cc05f6: Status 404 returned error can't find the container with id b7031cc027c13cd7a843ae57dab930f18b02c7a300228cc8ce7c9502f2cc05f6
Sep 09 21:10:51.946248 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:51.946208    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx" event={"ID":"5552cc45-a271-440e-b925-9c4ad819af36","Type":"ContainerStarted","Data":"1068211850df53347dcad5f10b0e2958c85baa3afb4cd15dd590cf0f4ae6477e"}
Sep 09 21:10:51.946711 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:51.946257    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx" event={"ID":"5552cc45-a271-440e-b925-9c4ad819af36","Type":"ContainerStarted","Data":"b7031cc027c13cd7a843ae57dab930f18b02c7a300228cc8ce7c9502f2cc05f6"}
Sep 09 21:10:51.964314 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:51.964246    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx" podStartSLOduration=1.964225042 podStartE2EDuration="1.964225042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:51.963158471 +0000 UTC m=+1345.933725186" watchObservedRunningTime="2026-09-09 21:10:51.964225042 +0000 UTC m=+1345.934791759"
Sep 09 21:10:56.917188 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:10:56.917147    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:11:00.821017 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:00.820981    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:11:00.821017 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:00.821028    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:11:00.834222 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:00.834193    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:11:01.013421 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:01.013393    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:11:08.093603 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:08.093553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:08.886423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:08.886384    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7601-d2c8c8"]
Sep 09 21:11:08.890119 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:08.890103    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-d2c8c8"
Sep 09 21:11:08.892479 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:08.892459    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cmlrc\""
Sep 09 21:11:08.900791 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:08.900759    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7601-d2c8c8"]
Sep 09 21:11:08.998159 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:08.998127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz7vd\" (UniqueName: \"kubernetes.io/projected/770ce75c-b014-421c-957a-0b3796ad06b6-kube-api-access-hz7vd\") pod \"test-curl-7601-d2c8c8\" (UID: \"770ce75c-b014-421c-957a-0b3796ad06b6\") " pod="opendatahub/test-curl-7601-d2c8c8"
Sep 09 21:11:09.099373 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:09.099343    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz7vd\" (UniqueName: \"kubernetes.io/projected/770ce75c-b014-421c-957a-0b3796ad06b6-kube-api-access-hz7vd\") pod \"test-curl-7601-d2c8c8\" (UID: \"770ce75c-b014-421c-957a-0b3796ad06b6\") " pod="opendatahub/test-curl-7601-d2c8c8"
Sep 09 21:11:09.107320 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:09.107291    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz7vd\" (UniqueName: \"kubernetes.io/projected/770ce75c-b014-421c-957a-0b3796ad06b6-kube-api-access-hz7vd\") pod \"test-curl-7601-d2c8c8\" (UID: \"770ce75c-b014-421c-957a-0b3796ad06b6\") " pod="opendatahub/test-curl-7601-d2c8c8"
Sep 09 21:11:09.200493 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:09.200454    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-d2c8c8"
Sep 09 21:11:09.474253 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:09.474218    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7601-d2c8c8"]
Sep 09 21:11:09.475614 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:11:09.475586    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod770ce75c_b014_421c_957a_0b3796ad06b6.slice/crio-221beda665a2618ae92a39da594572384b4f74ba49428da3063909cc30020395 WatchSource:0}: Error finding container 221beda665a2618ae92a39da594572384b4f74ba49428da3063909cc30020395: Status 404 returned error can't find the container with id 221beda665a2618ae92a39da594572384b4f74ba49428da3063909cc30020395
Sep 09 21:11:10.059165 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.059133    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7601-d2c8c8" event={"ID":"770ce75c-b014-421c-957a-0b3796ad06b6","Type":"ContainerStarted","Data":"f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58"}
Sep 09 21:11:10.059340 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.059175    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7601-d2c8c8" event={"ID":"770ce75c-b014-421c-957a-0b3796ad06b6","Type":"ContainerStarted","Data":"221beda665a2618ae92a39da594572384b4f74ba49428da3063909cc30020395"}
Sep 09 21:11:10.075567 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.075520    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7601-d2c8c8" podStartSLOduration=2.07550482 podStartE2EDuration="2.07550482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:10.073744194 +0000 UTC m=+1364.044310914" watchObservedRunningTime="2026-09-09 21:11:10.07550482 +0000 UTC m=+1364.046071536"
Sep 09 21:11:10.672933 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.672897    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7601-d2c8c8"]
Sep 09 21:11:10.677415 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.677381    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7601-d2c8c8"]
Sep 09 21:11:10.793532 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.793444    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7601-850d8e"]
Sep 09 21:11:10.793901 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.793887    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="770ce75c-b014-421c-957a-0b3796ad06b6" containerName="test-curl-7601-d2c8c8"
Sep 09 21:11:10.793945 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.793903    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="770ce75c-b014-421c-957a-0b3796ad06b6" containerName="test-curl-7601-d2c8c8"
Sep 09 21:11:10.793995 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.793987    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="770ce75c-b014-421c-957a-0b3796ad06b6" containerName="test-curl-7601-d2c8c8"
Sep 09 21:11:10.797347 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.797323    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:10.800517 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.800494    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7601-850d8e"]
Sep 09 21:11:10.916858 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:10.916820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrvd5\" (UniqueName: \"kubernetes.io/projected/3af55149-60fd-4c55-89ef-5b8a0af60159-kube-api-access-nrvd5\") pod \"test-curl-7601-850d8e\" (UID: \"3af55149-60fd-4c55-89ef-5b8a0af60159\") " pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:11.018216 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:11.018184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrvd5\" (UniqueName: \"kubernetes.io/projected/3af55149-60fd-4c55-89ef-5b8a0af60159-kube-api-access-nrvd5\") pod \"test-curl-7601-850d8e\" (UID: \"3af55149-60fd-4c55-89ef-5b8a0af60159\") " pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:11.026443 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:11.026420    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrvd5\" (UniqueName: \"kubernetes.io/projected/3af55149-60fd-4c55-89ef-5b8a0af60159-kube-api-access-nrvd5\") pod \"test-curl-7601-850d8e\" (UID: \"3af55149-60fd-4c55-89ef-5b8a0af60159\") " pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:11.065891 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:11.065812    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7601-d2c8c8" container="test-curl-7601-d2c8c8"
Sep 09 21:11:11.108599 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:11.108572    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:11.383649 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:11.383619    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7601-850d8e"]
Sep 09 21:11:11.385224 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:11:11.385202    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3af55149_60fd_4c55_89ef_5b8a0af60159.slice/crio-e244f509ccb98dfc777ee23d95e0895bfde36739d3679df8cbcdc90f288b3ec1 WatchSource:0}: Error finding container e244f509ccb98dfc777ee23d95e0895bfde36739d3679df8cbcdc90f288b3ec1: Status 404 returned error can't find the container with id e244f509ccb98dfc777ee23d95e0895bfde36739d3679df8cbcdc90f288b3ec1
Sep 09 21:11:12.073453 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.073420    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7601-850d8e" event={"ID":"3af55149-60fd-4c55-89ef-5b8a0af60159","Type":"ContainerStarted","Data":"fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515"}
Sep 09 21:11:12.073922 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.073463    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7601-850d8e" event={"ID":"3af55149-60fd-4c55-89ef-5b8a0af60159","Type":"ContainerStarted","Data":"e244f509ccb98dfc777ee23d95e0895bfde36739d3679df8cbcdc90f288b3ec1"}
Sep 09 21:11:12.073922 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.073544    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7601-d2c8c8" podUID="770ce75c-b014-421c-957a-0b3796ad06b6" containerName="test-curl-7601-d2c8c8" containerID="cri-o://f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58" gracePeriod=2
Sep 09 21:11:12.087419 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.087367    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7601-850d8e" podStartSLOduration=2.087348264 podStartE2EDuration="2.087348264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:12.087122105 +0000 UTC m=+1366.057688822" watchObservedRunningTime="2026-09-09 21:11:12.087348264 +0000 UTC m=+1366.057914981"
Sep 09 21:11:12.089065 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:12.089039    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7601-d2c8c8\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="770ce75c-b014-421c-957a-0b3796ad06b6" pod="opendatahub/test-curl-7601-d2c8c8"
Sep 09 21:11:12.673626 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.673593    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7601-850d8e"]
Sep 09 21:11:12.680266 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.680238    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7601-850d8e"]
Sep 09 21:11:12.793810 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.793732    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7601-55ec6b"]
Sep 09 21:11:12.794205 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.794192    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af55149-60fd-4c55-89ef-5b8a0af60159" containerName="test-curl-7601-850d8e"
Sep 09 21:11:12.794262 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.794207    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af55149-60fd-4c55-89ef-5b8a0af60159" containerName="test-curl-7601-850d8e"
Sep 09 21:11:12.794298 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.794291    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af55149-60fd-4c55-89ef-5b8a0af60159" containerName="test-curl-7601-850d8e"
Sep 09 21:11:12.797544 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.797529    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:12.801859 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.801833    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7601-55ec6b"]
Sep 09 21:11:12.936020 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:12.935979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dplk\" (UniqueName: \"kubernetes.io/projected/6ebfbd3f-b5c2-4784-8160-5feb468d9e97-kube-api-access-4dplk\") pod \"test-curl-7601-55ec6b\" (UID: \"6ebfbd3f-b5c2-4784-8160-5feb468d9e97\") " pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:13.036560 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:13.036516    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dplk\" (UniqueName: \"kubernetes.io/projected/6ebfbd3f-b5c2-4784-8160-5feb468d9e97-kube-api-access-4dplk\") pod \"test-curl-7601-55ec6b\" (UID: \"6ebfbd3f-b5c2-4784-8160-5feb468d9e97\") " pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:13.045129 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:13.045053    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dplk\" (UniqueName: \"kubernetes.io/projected/6ebfbd3f-b5c2-4784-8160-5feb468d9e97-kube-api-access-4dplk\") pod \"test-curl-7601-55ec6b\" (UID: \"6ebfbd3f-b5c2-4784-8160-5feb468d9e97\") " pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:13.080194 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:13.080165    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7601-850d8e" container="test-curl-7601-850d8e"
Sep 09 21:11:13.107858 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:13.107829    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:13.402836 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:13.402811    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7601-55ec6b"]
Sep 09 21:11:13.404414 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:11:13.404383    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ebfbd3f_b5c2_4784_8160_5feb468d9e97.slice/crio-6620e7e50d0629f87e5f9a40922f9ae4276d0dd190e1abff76e3753081ad4023 WatchSource:0}: Error finding container 6620e7e50d0629f87e5f9a40922f9ae4276d0dd190e1abff76e3753081ad4023: Status 404 returned error can't find the container with id 6620e7e50d0629f87e5f9a40922f9ae4276d0dd190e1abff76e3753081ad4023
Sep 09 21:11:14.088711 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.088639    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7601-55ec6b" event={"ID":"6ebfbd3f-b5c2-4784-8160-5feb468d9e97","Type":"ContainerStarted","Data":"d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4"}
Sep 09 21:11:14.088711 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.088687    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7601-55ec6b" event={"ID":"6ebfbd3f-b5c2-4784-8160-5feb468d9e97","Type":"ContainerStarted","Data":"6620e7e50d0629f87e5f9a40922f9ae4276d0dd190e1abff76e3753081ad4023"}
Sep 09 21:11:14.089237 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.088840    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7601-850d8e" podUID="3af55149-60fd-4c55-89ef-5b8a0af60159" containerName="test-curl-7601-850d8e" containerID="cri-o://fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515" gracePeriod=2
Sep 09 21:11:14.103215 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.103169    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7601-55ec6b" podStartSLOduration=2.103151449 podStartE2EDuration="2.103151449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:11:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:14.101881787 +0000 UTC m=+1368.072448504" watchObservedRunningTime="2026-09-09 21:11:14.103151449 +0000 UTC m=+1368.073718166"
Sep 09 21:11:14.103942 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:14.103907    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7601-850d8e\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="3af55149-60fd-4c55-89ef-5b8a0af60159" pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:14.231018 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.230986    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-d2c8c8"
Sep 09 21:11:14.233176 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:14.233152    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7601-850d8e\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="3af55149-60fd-4c55-89ef-5b8a0af60159" pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:14.349496 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.349404    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz7vd\" (UniqueName: \"kubernetes.io/projected/770ce75c-b014-421c-957a-0b3796ad06b6-kube-api-access-hz7vd\") pod \"770ce75c-b014-421c-957a-0b3796ad06b6\" (UID: \"770ce75c-b014-421c-957a-0b3796ad06b6\") "
Sep 09 21:11:14.351697 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.351673    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/770ce75c-b014-421c-957a-0b3796ad06b6-kube-api-access-hz7vd" (OuterVolumeSpecName: "kube-api-access-hz7vd") pod "770ce75c-b014-421c-957a-0b3796ad06b6" (UID: "770ce75c-b014-421c-957a-0b3796ad06b6"). InnerVolumeSpecName "kube-api-access-hz7vd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:11:14.450674 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.450638    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz7vd\" (UniqueName: \"kubernetes.io/projected/770ce75c-b014-421c-957a-0b3796ad06b6-kube-api-access-hz7vd\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:11:14.566397 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.566356    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="770ce75c-b014-421c-957a-0b3796ad06b6" path="/var/lib/kubelet/pods/770ce75c-b014-421c-957a-0b3796ad06b6/volumes"
Sep 09 21:11:14.703017 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.702980    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7601-55ec6b"]
Sep 09 21:11:14.707818 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.707788    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7601-55ec6b"]
Sep 09 21:11:14.819806 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.819770    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7601-5d2124"]
Sep 09 21:11:14.820271 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.820254    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" containerName="test-curl-7601-55ec6b"
Sep 09 21:11:14.820271 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.820271    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" containerName="test-curl-7601-55ec6b"
Sep 09 21:11:14.820437 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.820347    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" containerName="test-curl-7601-55ec6b"
Sep 09 21:11:14.823721 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.823695    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:14.830799 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.830773    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7601-5d2124"]
Sep 09 21:11:14.955634 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:14.955548    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4dbb\" (UniqueName: \"kubernetes.io/projected/fbc81b06-e76f-4e26-ab2b-bb8e6435a743-kube-api-access-v4dbb\") pod \"test-curl-7601-5d2124\" (UID: \"fbc81b06-e76f-4e26-ab2b-bb8e6435a743\") " pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:15.056735 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.056707    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4dbb\" (UniqueName: \"kubernetes.io/projected/fbc81b06-e76f-4e26-ab2b-bb8e6435a743-kube-api-access-v4dbb\") pod \"test-curl-7601-5d2124\" (UID: \"fbc81b06-e76f-4e26-ab2b-bb8e6435a743\") " pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:15.064429 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.064409    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4dbb\" (UniqueName: \"kubernetes.io/projected/fbc81b06-e76f-4e26-ab2b-bb8e6435a743-kube-api-access-v4dbb\") pod \"test-curl-7601-5d2124\" (UID: \"fbc81b06-e76f-4e26-ab2b-bb8e6435a743\") " pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:15.095809 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.095773    2571 generic.go:356] "Generic (PLEG): container finished" podID="770ce75c-b014-421c-957a-0b3796ad06b6" containerID="f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58" exitCode=137
Sep 09 21:11:15.096242 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.095830    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-d2c8c8"
Sep 09 21:11:15.096242 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.095866    2571 scope.go:117] "RemoveContainer" containerID="f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58"
Sep 09 21:11:15.096242 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:15.096119    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7601-55ec6b" container="test-curl-7601-55ec6b"
Sep 09 21:11:15.105432 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.105414    2571 scope.go:117] "RemoveContainer" containerID="f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58"
Sep 09 21:11:15.105711 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:15.105696    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58\": container with ID starting with f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58 not found: ID does not exist" containerID="f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58"
Sep 09 21:11:15.105759 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.105720    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58"} err="failed to get container status \"f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58\": rpc error: code = NotFound desc = could not find container \"f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58\": container with ID starting with f8e594a7e418dd6fc2355e15ceba4de3bcc275ba00aa6d9bdbb34c6424562f58 not found: ID does not exist"
Sep 09 21:11:15.134658 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.134623    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:15.409763 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.409736    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7601-5d2124"]
Sep 09 21:11:15.411347 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:11:15.411318    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbc81b06_e76f_4e26_ab2b_bb8e6435a743.slice/crio-41e30b6f9dffbff7bb15aa5f8d425c8320620b2c246941d65fc0870339c8f61c WatchSource:0}: Error finding container 41e30b6f9dffbff7bb15aa5f8d425c8320620b2c246941d65fc0870339c8f61c: Status 404 returned error can't find the container with id 41e30b6f9dffbff7bb15aa5f8d425c8320620b2c246941d65fc0870339c8f61c
Sep 09 21:11:15.608295 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.608221    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:15.666772 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:15.666660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:16.106521 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.106454    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7601-5d2124" event={"ID":"fbc81b06-e76f-4e26-ab2b-bb8e6435a743","Type":"ContainerStarted","Data":"9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2"}
Sep 09 21:11:16.106521 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.106494    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7601-5d2124" event={"ID":"fbc81b06-e76f-4e26-ab2b-bb8e6435a743","Type":"ContainerStarted","Data":"41e30b6f9dffbff7bb15aa5f8d425c8320620b2c246941d65fc0870339c8f61c"}
Sep 09 21:11:16.107763 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.107737    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7601-55ec6b" podUID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" containerName="test-curl-7601-55ec6b" containerID="cri-o://d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4" gracePeriod=2
Sep 09 21:11:16.124573 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:16.124539    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7601-55ec6b\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:16.125558 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.125518    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7601-5d2124" podStartSLOduration=2.1255033 podStartE2EDuration="2.1255033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:16.122596903 +0000 UTC m=+1370.093163621" watchObservedRunningTime="2026-09-09 21:11:16.1255033 +0000 UTC m=+1370.096070056"
Sep 09 21:11:16.273304 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.273283    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:16.276670 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:16.276648    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7601-55ec6b\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:16.371144 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.371040    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrvd5\" (UniqueName: \"kubernetes.io/projected/3af55149-60fd-4c55-89ef-5b8a0af60159-kube-api-access-nrvd5\") pod \"3af55149-60fd-4c55-89ef-5b8a0af60159\" (UID: \"3af55149-60fd-4c55-89ef-5b8a0af60159\") "
Sep 09 21:11:16.373168 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.373137    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af55149-60fd-4c55-89ef-5b8a0af60159-kube-api-access-nrvd5" (OuterVolumeSpecName: "kube-api-access-nrvd5") pod "3af55149-60fd-4c55-89ef-5b8a0af60159" (UID: "3af55149-60fd-4c55-89ef-5b8a0af60159"). InnerVolumeSpecName "kube-api-access-nrvd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:11:16.472227 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.472206    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrvd5\" (UniqueName: \"kubernetes.io/projected/3af55149-60fd-4c55-89ef-5b8a0af60159-kube-api-access-nrvd5\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:11:16.566124 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:16.566070    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7601-55ec6b\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:16.567703 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.567679    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af55149-60fd-4c55-89ef-5b8a0af60159" path="/var/lib/kubelet/pods/3af55149-60fd-4c55-89ef-5b8a0af60159/volumes"
Sep 09 21:11:16.675230 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.675146    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7601-5d2124"]
Sep 09 21:11:16.679987 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:16.679953    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7601-5d2124"]
Sep 09 21:11:17.114622 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:17.114586    2571 generic.go:356] "Generic (PLEG): container finished" podID="3af55149-60fd-4c55-89ef-5b8a0af60159" containerID="fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515" exitCode=137
Sep 09 21:11:17.115166 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:17.114630    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-850d8e"
Sep 09 21:11:17.115166 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:17.114690    2571 scope.go:117] "RemoveContainer" containerID="fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515"
Sep 09 21:11:17.115166 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:17.114849    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7601-5d2124" container="test-curl-7601-5d2124"
Sep 09 21:11:17.124513 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:17.124496    2571 scope.go:117] "RemoveContainer" containerID="fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515"
Sep 09 21:11:17.124775 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:17.124756    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515\": container with ID starting with fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515 not found: ID does not exist" containerID="fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515"
Sep 09 21:11:17.124857 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:17.124788    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515"} err="failed to get container status \"fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515\": rpc error: code = NotFound desc = could not find container \"fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515\": container with ID starting with fbf4516e83c68d7e443d7c8c58f24587f79f42ca33fce486fe8fe079b55cf515 not found: ID does not exist"
Sep 09 21:11:17.284817 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:17.284784    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:17.614834 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:17.614800    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:17.905100 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:17.904979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:18.122213 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:18.122176    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7601-5d2124" podUID="fbc81b06-e76f-4e26-ab2b-bb8e6435a743" containerName="test-curl-7601-5d2124" containerID="cri-o://9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2" gracePeriod=2
Sep 09 21:11:18.125274 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:18.125248    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7601-5d2124\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="fbc81b06-e76f-4e26-ab2b-bb8e6435a743" pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:18.300924 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:18.300900    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:18.304395 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:18.304370    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7601-5d2124\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="fbc81b06-e76f-4e26-ab2b-bb8e6435a743" pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:18.391970 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:18.391942    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dplk\" (UniqueName: \"kubernetes.io/projected/6ebfbd3f-b5c2-4784-8160-5feb468d9e97-kube-api-access-4dplk\") pod \"6ebfbd3f-b5c2-4784-8160-5feb468d9e97\" (UID: \"6ebfbd3f-b5c2-4784-8160-5feb468d9e97\") "
Sep 09 21:11:18.394219 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:18.394188    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ebfbd3f-b5c2-4784-8160-5feb468d9e97-kube-api-access-4dplk" (OuterVolumeSpecName: "kube-api-access-4dplk") pod "6ebfbd3f-b5c2-4784-8160-5feb468d9e97" (UID: "6ebfbd3f-b5c2-4784-8160-5feb468d9e97"). InnerVolumeSpecName "kube-api-access-4dplk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:11:18.493794 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:18.493762    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dplk\" (UniqueName: \"kubernetes.io/projected/6ebfbd3f-b5c2-4784-8160-5feb468d9e97-kube-api-access-4dplk\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:11:18.566833 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:18.566801    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" path="/var/lib/kubelet/pods/6ebfbd3f-b5c2-4784-8160-5feb468d9e97/volumes"
Sep 09 21:11:19.129376 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:19.129341    2571 generic.go:356] "Generic (PLEG): container finished" podID="6ebfbd3f-b5c2-4784-8160-5feb468d9e97" containerID="d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4" exitCode=137
Sep 09 21:11:19.129762 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:19.129384    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-55ec6b"
Sep 09 21:11:19.129762 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:19.129445    2571 scope.go:117] "RemoveContainer" containerID="d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4"
Sep 09 21:11:19.139351 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:19.139331    2571 scope.go:117] "RemoveContainer" containerID="d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4"
Sep 09 21:11:19.139685 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:19.139666    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4\": container with ID starting with d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4 not found: ID does not exist" containerID="d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4"
Sep 09 21:11:19.139741 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:19.139694    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4"} err="failed to get container status \"d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4\": rpc error: code = NotFound desc = could not find container \"d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4\": container with ID starting with d9df9e554eb3824c83d8ad339e282fd7046e0c1f2a8d03d9990173391a8373b4 not found: ID does not exist"
Sep 09 21:11:20.316008 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:20.315977    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:20.331167 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:20.331135    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:20.413471 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:20.413386    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4dbb\" (UniqueName: \"kubernetes.io/projected/fbc81b06-e76f-4e26-ab2b-bb8e6435a743-kube-api-access-v4dbb\") pod \"fbc81b06-e76f-4e26-ab2b-bb8e6435a743\" (UID: \"fbc81b06-e76f-4e26-ab2b-bb8e6435a743\") "
Sep 09 21:11:20.415638 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:20.415614    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbc81b06-e76f-4e26-ab2b-bb8e6435a743-kube-api-access-v4dbb" (OuterVolumeSpecName: "kube-api-access-v4dbb") pod "fbc81b06-e76f-4e26-ab2b-bb8e6435a743" (UID: "fbc81b06-e76f-4e26-ab2b-bb8e6435a743"). InnerVolumeSpecName "kube-api-access-v4dbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:11:20.440462 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:20.440429    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:20.515180 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:20.515149    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4dbb\" (UniqueName: \"kubernetes.io/projected/fbc81b06-e76f-4e26-ab2b-bb8e6435a743-kube-api-access-v4dbb\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:11:20.567394 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:20.567362    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbc81b06-e76f-4e26-ab2b-bb8e6435a743" path="/var/lib/kubelet/pods/fbc81b06-e76f-4e26-ab2b-bb8e6435a743/volumes"
Sep 09 21:11:21.145556 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.145517    2571 generic.go:356] "Generic (PLEG): container finished" podID="fbc81b06-e76f-4e26-ab2b-bb8e6435a743" containerID="9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2" exitCode=137
Sep 09 21:11:21.145750 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.145560    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7601-5d2124"
Sep 09 21:11:21.145750 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.145578    2571 scope.go:117] "RemoveContainer" containerID="9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2"
Sep 09 21:11:21.155203 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.155185    2571 scope.go:117] "RemoveContainer" containerID="9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2"
Sep 09 21:11:21.155474 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:11:21.155451    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2\": container with ID starting with 9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2 not found: ID does not exist" containerID="9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2"
Sep 09 21:11:21.155578 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.155480    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2"} err="failed to get container status \"9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2\": rpc error: code = NotFound desc = could not find container \"9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2\": container with ID starting with 9fff1b64b6c48c52a547ddd2cbf9948fc2d62739a6e6b3cc4abf18aa920ceaa2 not found: ID does not exist"
Sep 09 21:11:21.473640 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.473612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:21.576565 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.576529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:21.709582 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.709552    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:21.989635 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:21.989593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:22.076363 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.076329    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:22.113959 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.113929    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"]
Sep 09 21:11:22.114401 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.114386    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbc81b06-e76f-4e26-ab2b-bb8e6435a743" containerName="test-curl-7601-5d2124"
Sep 09 21:11:22.114462 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.114403    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbc81b06-e76f-4e26-ab2b-bb8e6435a743" containerName="test-curl-7601-5d2124"
Sep 09 21:11:22.114513 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.114497    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbc81b06-e76f-4e26-ab2b-bb8e6435a743" containerName="test-curl-7601-5d2124"
Sep 09 21:11:22.119343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.119321    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:22.125668 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.125631    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f8b5626b\""
Sep 09 21:11:22.126529 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.126508    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f8b5626b-dockercfg-nbk8t\""
Sep 09 21:11:22.130142 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.130116    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"]
Sep 09 21:11:22.232863 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.232828    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0099d3e-259e-497d-9c34-5a56449cc1ed-plugins-config-volume\") pod \"payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9\" (UID: \"e0099d3e-259e-497d-9c34-5a56449cc1ed\") " pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:22.233041 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.232874    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhpkf\" (UniqueName: \"kubernetes.io/projected/e0099d3e-259e-497d-9c34-5a56449cc1ed-kube-api-access-fhpkf\") pod \"payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9\" (UID: \"e0099d3e-259e-497d-9c34-5a56449cc1ed\") " pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:22.334337 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.334236    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0099d3e-259e-497d-9c34-5a56449cc1ed-plugins-config-volume\") pod \"payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9\" (UID: \"e0099d3e-259e-497d-9c34-5a56449cc1ed\") " pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:22.334337 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.334294    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhpkf\" (UniqueName: \"kubernetes.io/projected/e0099d3e-259e-497d-9c34-5a56449cc1ed-kube-api-access-fhpkf\") pod \"payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9\" (UID: \"e0099d3e-259e-497d-9c34-5a56449cc1ed\") " pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:22.334946 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.334926    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0099d3e-259e-497d-9c34-5a56449cc1ed-plugins-config-volume\") pod \"payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9\" (UID: \"e0099d3e-259e-497d-9c34-5a56449cc1ed\") " pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:22.345276 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.345245    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhpkf\" (UniqueName: \"kubernetes.io/projected/e0099d3e-259e-497d-9c34-5a56449cc1ed-kube-api-access-fhpkf\") pod \"payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9\" (UID: \"e0099d3e-259e-497d-9c34-5a56449cc1ed\") " pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:22.432273 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.432235    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:22.711022 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:22.710997    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"]
Sep 09 21:11:22.712693 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:11:22.712670    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0099d3e_259e_497d_9c34_5a56449cc1ed.slice/crio-882b8074e93490b6376296ca3f8876140bcf2d79808855b30b266ecc7bd8f0d4 WatchSource:0}: Error finding container 882b8074e93490b6376296ca3f8876140bcf2d79808855b30b266ecc7bd8f0d4: Status 404 returned error can't find the container with id 882b8074e93490b6376296ca3f8876140bcf2d79808855b30b266ecc7bd8f0d4
Sep 09 21:11:23.168957 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:23.168842    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9" event={"ID":"e0099d3e-259e-497d-9c34-5a56449cc1ed","Type":"ContainerStarted","Data":"e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b"}
Sep 09 21:11:23.169159 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:23.169118    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:23.169238 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:23.169187    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9" event={"ID":"e0099d3e-259e-497d-9c34-5a56449cc1ed","Type":"ContainerStarted","Data":"882b8074e93490b6376296ca3f8876140bcf2d79808855b30b266ecc7bd8f0d4"}
Sep 09 21:11:23.186242 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:23.186194    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9" podStartSLOduration=1.186181017 podStartE2EDuration="1.186181017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:23.184281069 +0000 UTC m=+1377.154847787" watchObservedRunningTime="2026-09-09 21:11:23.186181017 +0000 UTC m=+1377.156747734"
Sep 09 21:11:23.696276 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:23.696228    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:26.738351 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:26.738323    2571 scope.go:117] "RemoveContainer" containerID="23c3f156a2a9050bcefc6ca8fdb30757dd1ee3d5e0fc8c3a162eeaec1120b209"
Sep 09 21:11:34.178269 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:34.178240    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:11:45.337358 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:45.337324    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:49.242720 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:49.242684    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:11:55.734204 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:11:55.734169    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:12:14.429928 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.429890    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"]
Sep 09 21:12:14.433797 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.433778    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"
Sep 09 21:12:14.436186 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.436162    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:12:14.442046 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.442024    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"]
Sep 09 21:12:14.540389 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.540354    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:12:14.546511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.546477    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c600edf3-864b-43da-969e-27c67858db19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n\" (UID: \"c600edf3-864b-43da-969e-27c67858db19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"
Sep 09 21:12:14.647302 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.647271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c600edf3-864b-43da-969e-27c67858db19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n\" (UID: \"c600edf3-864b-43da-969e-27c67858db19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"
Sep 09 21:12:14.647950 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.647926    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c600edf3-864b-43da-969e-27c67858db19-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n\" (UID: \"c600edf3-864b-43da-969e-27c67858db19\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"
Sep 09 21:12:14.745514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:14.745472    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"
Sep 09 21:12:15.017271 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:15.017244    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"]
Sep 09 21:12:15.019558 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:12:15.019535    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc600edf3_864b_43da_969e_27c67858db19.slice/crio-886465053a41151d300240a5566f781b033867689d2ac58d54e618e017b7e6a4 WatchSource:0}: Error finding container 886465053a41151d300240a5566f781b033867689d2ac58d54e618e017b7e6a4: Status 404 returned error can't find the container with id 886465053a41151d300240a5566f781b033867689d2ac58d54e618e017b7e6a4
Sep 09 21:12:15.505681 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:15.505639    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n" event={"ID":"c600edf3-864b-43da-969e-27c67858db19","Type":"ContainerStarted","Data":"edb9320f1ff1029a946e18f264b662352a08936af6b8fa6ce57a8c695a33825f"}
Sep 09 21:12:15.506101 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:15.505731    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n" event={"ID":"c600edf3-864b-43da-969e-27c67858db19","Type":"ContainerStarted","Data":"886465053a41151d300240a5566f781b033867689d2ac58d54e618e017b7e6a4"}
Sep 09 21:12:15.519584 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:15.519516    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n" podStartSLOduration=1.519496833 podStartE2EDuration="1.519496833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:12:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:12:15.519073271 +0000 UTC m=+1429.489639990" watchObservedRunningTime="2026-09-09 21:12:15.519496833 +0000 UTC m=+1429.490063551"
Sep 09 21:12:16.513930 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:16.513893    2571 generic.go:356] "Generic (PLEG): container finished" podID="c600edf3-864b-43da-969e-27c67858db19" containerID="edb9320f1ff1029a946e18f264b662352a08936af6b8fa6ce57a8c695a33825f" exitCode=0
Sep 09 21:12:16.514372 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:16.513990    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n" event={"ID":"c600edf3-864b-43da-969e-27c67858db19","Type":"ContainerDied","Data":"edb9320f1ff1029a946e18f264b662352a08936af6b8fa6ce57a8c695a33825f"}
Sep 09 21:12:17.655162 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:17.655137    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"
Sep 09 21:12:17.781343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:17.780542    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c600edf3-864b-43da-969e-27c67858db19-maas-api-service-ca\") pod \"c600edf3-864b-43da-969e-27c67858db19\" (UID: \"c600edf3-864b-43da-969e-27c67858db19\") "
Sep 09 21:12:17.782107 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:17.781861    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c600edf3-864b-43da-969e-27c67858db19-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c600edf3-864b-43da-969e-27c67858db19" (UID: "c600edf3-864b-43da-969e-27c67858db19"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:12:17.882561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:17.882527    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c600edf3-864b-43da-969e-27c67858db19-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:12:18.529481 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:18.529446    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n" event={"ID":"c600edf3-864b-43da-969e-27c67858db19","Type":"ContainerDied","Data":"886465053a41151d300240a5566f781b033867689d2ac58d54e618e017b7e6a4"}
Sep 09 21:12:18.529481 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:18.529476    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"
Sep 09 21:12:18.529752 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:18.529525    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="886465053a41151d300240a5566f781b033867689d2ac58d54e618e017b7e6a4"
Sep 09 21:12:30.005559 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.005508    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:12:30.016801 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.016768    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"]
Sep 09 21:12:30.017010 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.016970    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9" podUID="e0099d3e-259e-497d-9c34-5a56449cc1ed" containerName="payload-processing" containerID="cri-o://e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b" gracePeriod=30
Sep 09 21:12:30.102242 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.102208    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:12:30.214763 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.214732    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:12:30.256414 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.256337    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"]
Sep 09 21:12:30.265857 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.265818    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8b5626b-9cs4n"]
Sep 09 21:12:30.277201 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.277177    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:12:30.339809 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.339770    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:12:30.402820 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.402794    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhpkf\" (UniqueName: \"kubernetes.io/projected/e0099d3e-259e-497d-9c34-5a56449cc1ed-kube-api-access-fhpkf\") pod \"e0099d3e-259e-497d-9c34-5a56449cc1ed\" (UID: \"e0099d3e-259e-497d-9c34-5a56449cc1ed\") "
Sep 09 21:12:30.402981 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.402875    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0099d3e-259e-497d-9c34-5a56449cc1ed-plugins-config-volume\") pod \"e0099d3e-259e-497d-9c34-5a56449cc1ed\" (UID: \"e0099d3e-259e-497d-9c34-5a56449cc1ed\") "
Sep 09 21:12:30.403232 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.403211    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0099d3e-259e-497d-9c34-5a56449cc1ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0099d3e-259e-497d-9c34-5a56449cc1ed" (UID: "e0099d3e-259e-497d-9c34-5a56449cc1ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:12:30.405107 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.405072    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0099d3e-259e-497d-9c34-5a56449cc1ed-kube-api-access-fhpkf" (OuterVolumeSpecName: "kube-api-access-fhpkf") pod "e0099d3e-259e-497d-9c34-5a56449cc1ed" (UID: "e0099d3e-259e-497d-9c34-5a56449cc1ed"). InnerVolumeSpecName "kube-api-access-fhpkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:12:30.503627 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.503588    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhpkf\" (UniqueName: \"kubernetes.io/projected/e0099d3e-259e-497d-9c34-5a56449cc1ed-kube-api-access-fhpkf\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:12:30.503627 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.503618    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0099d3e-259e-497d-9c34-5a56449cc1ed-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:12:30.566410 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.566319    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c600edf3-864b-43da-969e-27c67858db19" path="/var/lib/kubelet/pods/c600edf3-864b-43da-969e-27c67858db19/volumes"
Sep 09 21:12:30.603918 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.603878    2571 generic.go:356] "Generic (PLEG): container finished" podID="e0099d3e-259e-497d-9c34-5a56449cc1ed" containerID="e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b" exitCode=0
Sep 09 21:12:30.604115 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.603936    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9" event={"ID":"e0099d3e-259e-497d-9c34-5a56449cc1ed","Type":"ContainerDied","Data":"e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b"}
Sep 09 21:12:30.604115 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.603977    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"
Sep 09 21:12:30.604115 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.603990    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9" event={"ID":"e0099d3e-259e-497d-9c34-5a56449cc1ed","Type":"ContainerDied","Data":"882b8074e93490b6376296ca3f8876140bcf2d79808855b30b266ecc7bd8f0d4"}
Sep 09 21:12:30.604115 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.604022    2571 scope.go:117] "RemoveContainer" containerID="e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b"
Sep 09 21:12:30.614072 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.614048    2571 scope.go:117] "RemoveContainer" containerID="e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b"
Sep 09 21:12:30.614393 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:12:30.614362    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b\": container with ID starting with e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b not found: ID does not exist" containerID="e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b"
Sep 09 21:12:30.614482 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.614409    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b"} err="failed to get container status \"e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b\": rpc error: code = NotFound desc = could not find container \"e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b\": container with ID starting with e01984d170c5ffe9ae27c9adbbd41f2024b897e0f6b4275e6b022d8f07a2ed6b not found: ID does not exist"
Sep 09 21:12:30.622652 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.622621    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"]
Sep 09 21:12:30.626790 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:30.626765    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f8b5626b-66cd4f74f7-dcwz9"]
Sep 09 21:12:32.572698 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:32.572664    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0099d3e-259e-497d-9c34-5a56449cc1ed" path="/var/lib/kubelet/pods/e0099d3e-259e-497d-9c34-5a56449cc1ed/volumes"
Sep 09 21:12:33.328329 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:33.328291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:12:40.011925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:12:40.011882    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:12.432178 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:12.432134    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:14.502024 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:14.501987    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:15.786533 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:15.786501    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:21.699128 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:21.699066    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:21.770413 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:21.770368    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:21.973827 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:21.973737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:23.271088 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:23.271048    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:23.368726 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:23.368693    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:23.468567 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:23.468526    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:24.108628 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:24.108592    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:24.208578 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:24.208542    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:24.305229 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:24.305192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:25.572796 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:25.572756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:26.789441 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:26.789408    2571 scope.go:117] "RemoveContainer" containerID="541c806667720be5edce6e439b5c2050a017345ace200b9df6ef77abb15b0ae5"
Sep 09 21:13:26.799519 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:26.799491    2571 scope.go:117] "RemoveContainer" containerID="c720aa1af59ac87b964329dcf9aa6fd97dd004125591180a0a17cc2c63219315"
Sep 09 21:13:31.517783 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:31.517743    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:32.417342 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.417296    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7604-12663f"]
Sep 09 21:13:32.417789 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.417774    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c600edf3-864b-43da-969e-27c67858db19" containerName="revoke-keys"
Sep 09 21:13:32.417836 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.417791    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c600edf3-864b-43da-969e-27c67858db19" containerName="revoke-keys"
Sep 09 21:13:32.417836 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.417808    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0099d3e-259e-497d-9c34-5a56449cc1ed" containerName="payload-processing"
Sep 09 21:13:32.417836 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.417814    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0099d3e-259e-497d-9c34-5a56449cc1ed" containerName="payload-processing"
Sep 09 21:13:32.417944 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.417887    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0099d3e-259e-497d-9c34-5a56449cc1ed" containerName="payload-processing"
Sep 09 21:13:32.417944 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.417899    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c600edf3-864b-43da-969e-27c67858db19" containerName="revoke-keys"
Sep 09 21:13:32.422621 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.422592    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:32.424898 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.424872    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cmlrc\""
Sep 09 21:13:32.426254 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.426231    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-12663f"]
Sep 09 21:13:32.520036 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.520005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pgk5\" (UniqueName: \"kubernetes.io/projected/14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9-kube-api-access-6pgk5\") pod \"test-curl-7604-12663f\" (UID: \"14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9\") " pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:32.621465 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.621426    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pgk5\" (UniqueName: \"kubernetes.io/projected/14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9-kube-api-access-6pgk5\") pod \"test-curl-7604-12663f\" (UID: \"14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9\") " pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:32.629196 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.629169    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pgk5\" (UniqueName: \"kubernetes.io/projected/14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9-kube-api-access-6pgk5\") pod \"test-curl-7604-12663f\" (UID: \"14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9\") " pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:32.733481 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:32.733434    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:33.004017 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:33.003987    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-12663f"]
Sep 09 21:13:33.005100 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:13:33.005046    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14c4e7e0_9a44_4c55_8b4c_94ab277a7bd9.slice/crio-972ad007e906a17cbd884d62991b431a4a5fdd1f4de4bd4f46f055127dc92922 WatchSource:0}: Error finding container 972ad007e906a17cbd884d62991b431a4a5fdd1f4de4bd4f46f055127dc92922: Status 404 returned error can't find the container with id 972ad007e906a17cbd884d62991b431a4a5fdd1f4de4bd4f46f055127dc92922
Sep 09 21:13:33.254198 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:33.254101    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:33.997972 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:33.997941    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-12663f" event={"ID":"14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9","Type":"ContainerStarted","Data":"df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591"}
Sep 09 21:13:33.997972 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:33.997977    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-12663f" event={"ID":"14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9","Type":"ContainerStarted","Data":"972ad007e906a17cbd884d62991b431a4a5fdd1f4de4bd4f46f055127dc92922"}
Sep 09 21:13:34.012391 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:34.012345    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7604-12663f" podStartSLOduration=2.012330081 podStartE2EDuration="2.012330081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:13:34.010995951 +0000 UTC m=+1507.981562664" watchObservedRunningTime="2026-09-09 21:13:34.012330081 +0000 UTC m=+1507.982896798"
Sep 09 21:13:34.595229 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:34.595175    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7604-12663f"]
Sep 09 21:13:34.599119 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:34.599066    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7604-12663f"]
Sep 09 21:13:35.004398 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:13:35.004369    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7604-12663f" container="test-curl-7604-12663f"
Sep 09 21:13:35.938269 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:35.938232    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"]
Sep 09 21:13:35.949262 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:35.949232    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:36.011106 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:36.011047    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7604-12663f" podUID="14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" containerName="test-curl-7604-12663f" containerID="cri-o://df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591" gracePeriod=2
Sep 09 21:13:36.013188 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:13:36.013166    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-12663f\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:36.571893 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:13:36.571860    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-12663f\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:37.123893 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:37.123856    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:38.198816 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.198787    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:38.275650 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.275551    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pgk5\" (UniqueName: \"kubernetes.io/projected/14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9-kube-api-access-6pgk5\") pod \"14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9\" (UID: \"14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9\") "
Sep 09 21:13:38.277840 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.277814    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9-kube-api-access-6pgk5" (OuterVolumeSpecName: "kube-api-access-6pgk5") pod "14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" (UID: "14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9"). InnerVolumeSpecName "kube-api-access-6pgk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:13:38.377258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.377216    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pgk5\" (UniqueName: \"kubernetes.io/projected/14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9-kube-api-access-6pgk5\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:13:38.461475 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.461440    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:38.527648 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.527561    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:38.567061 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.567029    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" path="/var/lib/kubelet/pods/14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9/volumes"
Sep 09 21:13:38.634970 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.634936    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:38.742320 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:38.742283    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:39.030731 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:39.030696    2571 generic.go:356] "Generic (PLEG): container finished" podID="14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" containerID="df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591" exitCode=137
Sep 09 21:13:39.030894 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:39.030738    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-12663f"
Sep 09 21:13:39.030894 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:39.030801    2571 scope.go:117] "RemoveContainer" containerID="df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591"
Sep 09 21:13:39.042601 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:39.042583    2571 scope.go:117] "RemoveContainer" containerID="df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591"
Sep 09 21:13:39.042872 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:13:39.042857    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591\": container with ID starting with df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591 not found: ID does not exist" containerID="df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591"
Sep 09 21:13:39.042913 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:39.042880    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591"} err="failed to get container status \"df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591\": rpc error: code = NotFound desc = could not find container \"df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591\": container with ID starting with df656bc146912c4e212ce004d360b4efd236dfb75658772bec3a326a51f91591 not found: ID does not exist"
Sep 09 21:13:41.738263 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:41.738229    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:49.181188 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:49.181103    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:49.408228 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:49.408192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:13:50.961400 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:50.961355    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx" podUID="5552cc45-a271-440e-b925-9c4ad819af36" containerName="main" containerID="cri-o://1068211850df53347dcad5f10b0e2958c85baa3afb4cd15dd590cf0f4ae6477e" gracePeriod=45
Sep 09 21:13:51.003385 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.003344    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx" podUID="5552cc45-a271-440e-b925-9c4ad819af36" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": dial tcp 10.132.0.79:8000: connect: connection refused"
Sep 09 21:13:51.109256 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.109214    2571 generic.go:356] "Generic (PLEG): container finished" podID="5552cc45-a271-440e-b925-9c4ad819af36" containerID="1068211850df53347dcad5f10b0e2958c85baa3afb4cd15dd590cf0f4ae6477e" exitCode=0
Sep 09 21:13:51.109435 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.109293    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx" event={"ID":"5552cc45-a271-440e-b925-9c4ad819af36","Type":"ContainerDied","Data":"1068211850df53347dcad5f10b0e2958c85baa3afb4cd15dd590cf0f4ae6477e"}
Sep 09 21:13:51.215284 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.215214    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:13:51.314209 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314171    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-home\") pod \"5552cc45-a271-440e-b925-9c4ad819af36\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") "
Sep 09 21:13:51.314423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314241    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5552cc45-a271-440e-b925-9c4ad819af36-tls-certs\") pod \"5552cc45-a271-440e-b925-9c4ad819af36\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") "
Sep 09 21:13:51.314423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314299    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntm9c\" (UniqueName: \"kubernetes.io/projected/5552cc45-a271-440e-b925-9c4ad819af36-kube-api-access-ntm9c\") pod \"5552cc45-a271-440e-b925-9c4ad819af36\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") "
Sep 09 21:13:51.314423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314332    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-tmp-dir\") pod \"5552cc45-a271-440e-b925-9c4ad819af36\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") "
Sep 09 21:13:51.314423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314348    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-dshm\") pod \"5552cc45-a271-440e-b925-9c4ad819af36\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") "
Sep 09 21:13:51.314423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314368    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-model-cache\") pod \"5552cc45-a271-440e-b925-9c4ad819af36\" (UID: \"5552cc45-a271-440e-b925-9c4ad819af36\") "
Sep 09 21:13:51.314704 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314521    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-home" (OuterVolumeSpecName: "home") pod "5552cc45-a271-440e-b925-9c4ad819af36" (UID: "5552cc45-a271-440e-b925-9c4ad819af36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:51.314704 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314648    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5552cc45-a271-440e-b925-9c4ad819af36" (UID: "5552cc45-a271-440e-b925-9c4ad819af36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:51.314704 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314697    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-home\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:13:51.314814 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.314695    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-model-cache" (OuterVolumeSpecName: "model-cache") pod "5552cc45-a271-440e-b925-9c4ad819af36" (UID: "5552cc45-a271-440e-b925-9c4ad819af36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:51.316863 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.316843    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5552cc45-a271-440e-b925-9c4ad819af36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5552cc45-a271-440e-b925-9c4ad819af36" (UID: "5552cc45-a271-440e-b925-9c4ad819af36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:13:51.316863 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.316853    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-dshm" (OuterVolumeSpecName: "dshm") pod "5552cc45-a271-440e-b925-9c4ad819af36" (UID: "5552cc45-a271-440e-b925-9c4ad819af36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:13:51.317011 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.316877    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5552cc45-a271-440e-b925-9c4ad819af36-kube-api-access-ntm9c" (OuterVolumeSpecName: "kube-api-access-ntm9c") pod "5552cc45-a271-440e-b925-9c4ad819af36" (UID: "5552cc45-a271-440e-b925-9c4ad819af36"). InnerVolumeSpecName "kube-api-access-ntm9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:13:51.415461 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.415423    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntm9c\" (UniqueName: \"kubernetes.io/projected/5552cc45-a271-440e-b925-9c4ad819af36-kube-api-access-ntm9c\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:13:51.415461 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.415454    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-tmp-dir\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:13:51.415461 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.415464    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-dshm\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:13:51.415461 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.415473    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5552cc45-a271-440e-b925-9c4ad819af36-model-cache\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:13:51.415741 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:51.415481    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5552cc45-a271-440e-b925-9c4ad819af36-tls-certs\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:13:52.117152 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:52.117127    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"
Sep 09 21:13:52.117583 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:52.117151    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx" event={"ID":"5552cc45-a271-440e-b925-9c4ad819af36","Type":"ContainerDied","Data":"b7031cc027c13cd7a843ae57dab930f18b02c7a300228cc8ce7c9502f2cc05f6"}
Sep 09 21:13:52.117583 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:52.117203    2571 scope.go:117] "RemoveContainer" containerID="1068211850df53347dcad5f10b0e2958c85baa3afb4cd15dd590cf0f4ae6477e"
Sep 09 21:13:52.141862 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:52.141834    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"]
Sep 09 21:13:52.147113 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:52.147066    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3d3cac/auth-test-model-3d3cac-kserve-54c6bbdb95-4p5gx"]
Sep 09 21:13:52.567142 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:13:52.567111    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5552cc45-a271-440e-b925-9c4ad819af36" path="/var/lib/kubelet/pods/5552cc45-a271-440e-b925-9c4ad819af36/volumes"
Sep 09 21:14:05.620638 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.620605    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"]
Sep 09 21:14:05.621292 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.621274    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5552cc45-a271-440e-b925-9c4ad819af36" containerName="main"
Sep 09 21:14:05.621389 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.621294    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5552cc45-a271-440e-b925-9c4ad819af36" containerName="main"
Sep 09 21:14:05.621389 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.621312    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" containerName="test-curl-7604-12663f"
Sep 09 21:14:05.621389 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.621320    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" containerName="test-curl-7604-12663f"
Sep 09 21:14:05.621554 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.621434    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="14c4e7e0-9a44-4c55-8b4c-94ab277a7bd9" containerName="test-curl-7604-12663f"
Sep 09 21:14:05.621554 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.621454    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5552cc45-a271-440e-b925-9c4ad819af36" containerName="main"
Sep 09 21:14:05.624729 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.624711    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.628099 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.628059    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8db5a7\"/\"sub-test-model-8db5a7-kserve-self-signed-certs\""
Sep 09 21:14:05.628224 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.628061    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8db5a7\"/\"kube-root-ca.crt\""
Sep 09 21:14:05.628224 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.628119    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8db5a7\"/\"openshift-service-ca.crt\""
Sep 09 21:14:05.628224 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.628158    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8db5a7\"/\"default-dockercfg-bf86r\""
Sep 09 21:14:05.636419 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.636393    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"]
Sep 09 21:14:05.653460 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.653425    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjh66\" (UniqueName: \"kubernetes.io/projected/d0bcfba8-d621-42a9-a40d-158cefa399bc-kube-api-access-kjh66\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.653673 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.653471    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-home\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.653673 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.653565    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-model-cache\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.653673 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.653602    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-tmp-dir\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.653673 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.653634    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-dshm\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.653856 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.653695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0bcfba8-d621-42a9-a40d-158cefa399bc-tls-certs\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.657846 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.657818    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:05.755213 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755170    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0bcfba8-d621-42a9-a40d-158cefa399bc-tls-certs\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.755425 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjh66\" (UniqueName: \"kubernetes.io/projected/d0bcfba8-d621-42a9-a40d-158cefa399bc-kube-api-access-kjh66\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.755425 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755321    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-home\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.755425 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-model-cache\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.755425 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-tmp-dir\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.755425 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755414    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-dshm\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.755736 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755716    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-model-cache\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.755811 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755788    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-home\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.755865 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.755807    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-tmp-dir\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.757784 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.757750    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-dshm\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.757894 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.757862    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0bcfba8-d621-42a9-a40d-158cefa399bc-tls-certs\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.763789 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.763769    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjh66\" (UniqueName: \"kubernetes.io/projected/d0bcfba8-d621-42a9-a40d-158cefa399bc-kube-api-access-kjh66\") pod \"sub-test-model-8db5a7-kserve-67d955cb65-27bsj\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") " pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:05.935788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:05.935696    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:06.212554 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:06.212521    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"]
Sep 09 21:14:06.213744 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:14:06.213719    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0bcfba8_d621_42a9_a40d_158cefa399bc.slice/crio-47859715e93587d417bc832ee33f5be64709e4bf4f7da416500fe5edcc84a651 WatchSource:0}: Error finding container 47859715e93587d417bc832ee33f5be64709e4bf4f7da416500fe5edcc84a651: Status 404 returned error can't find the container with id 47859715e93587d417bc832ee33f5be64709e4bf4f7da416500fe5edcc84a651
Sep 09 21:14:07.211004 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:07.210964    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj" event={"ID":"d0bcfba8-d621-42a9-a40d-158cefa399bc","Type":"ContainerStarted","Data":"f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038"}
Sep 09 21:14:07.211004 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:07.211013    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj" event={"ID":"d0bcfba8-d621-42a9-a40d-158cefa399bc","Type":"ContainerStarted","Data":"47859715e93587d417bc832ee33f5be64709e4bf4f7da416500fe5edcc84a651"}
Sep 09 21:14:07.232045 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:07.231992    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj" podStartSLOduration=2.231976022 podStartE2EDuration="2.231976022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:07.230463436 +0000 UTC m=+1541.201030154" watchObservedRunningTime="2026-09-09 21:14:07.231976022 +0000 UTC m=+1541.202542742"
Sep 09 21:14:09.527578 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:09.527545    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:10.366954 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:10.366917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:15.936573 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:15.936524    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:15.937007 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:15.936610    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:15.949380 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:15.949354    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:16.276955 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:16.276924    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:14:16.461315 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:16.461274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:16.560174 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:16.560041    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:16.858126 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:16.857991    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:17.175659 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:17.175561    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:17.348145 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:17.348113    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:20.186852 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:20.186811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:20.282721 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:20.282670    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:21.444877 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:21.444815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:21.554511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:21.554446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:21.902617 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:21.902502    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:22.009508 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.009475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:22.096317 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.096285    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"]
Sep 09 21:14:22.097114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.097056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:22.100546 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.100526    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:22.102919 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.102901    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-dc3099-dockercfg-zp7fg\""
Sep 09 21:14:22.103011 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.102898    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-dc3099\""
Sep 09 21:14:22.111346 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.111314    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"]
Sep 09 21:14:22.123281 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.123258    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"]
Sep 09 21:14:22.127902 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.127881    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:22.140778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.140746    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"]
Sep 09 21:14:22.219747 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.219720    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e85a3441-f45b-47a6-acd0-aea38b2df7bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-dc3099-69445df59-md827\" (UID: \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:22.219925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.219769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b3c213-f669-469c-aa17-aab28f9fc2c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs\" (UID: \"73b3c213-f669-469c-aa17-aab28f9fc2c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:22.219925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.219797    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs7sd\" (UniqueName: \"kubernetes.io/projected/e85a3441-f45b-47a6-acd0-aea38b2df7bf-kube-api-access-hs7sd\") pod \"payload-processing-e2e-api-a-dc3099-69445df59-md827\" (UID: \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:22.219925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.219905    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9m7t\" (UniqueName: \"kubernetes.io/projected/73b3c213-f669-469c-aa17-aab28f9fc2c2-kube-api-access-k9m7t\") pod \"payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs\" (UID: \"73b3c213-f669-469c-aa17-aab28f9fc2c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:22.321212 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.321179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9m7t\" (UniqueName: \"kubernetes.io/projected/73b3c213-f669-469c-aa17-aab28f9fc2c2-kube-api-access-k9m7t\") pod \"payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs\" (UID: \"73b3c213-f669-469c-aa17-aab28f9fc2c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:22.321411 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.321241    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e85a3441-f45b-47a6-acd0-aea38b2df7bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-dc3099-69445df59-md827\" (UID: \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:22.321411 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.321267    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b3c213-f669-469c-aa17-aab28f9fc2c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs\" (UID: \"73b3c213-f669-469c-aa17-aab28f9fc2c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:22.321411 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.321290    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs7sd\" (UniqueName: \"kubernetes.io/projected/e85a3441-f45b-47a6-acd0-aea38b2df7bf-kube-api-access-hs7sd\") pod \"payload-processing-e2e-api-a-dc3099-69445df59-md827\" (UID: \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:22.321978 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.321954    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b3c213-f669-469c-aa17-aab28f9fc2c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs\" (UID: \"73b3c213-f669-469c-aa17-aab28f9fc2c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:22.322110 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.321998    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e85a3441-f45b-47a6-acd0-aea38b2df7bf-plugins-config-volume\") pod \"payload-processing-e2e-api-a-dc3099-69445df59-md827\" (UID: \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:22.339755 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.339724    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9m7t\" (UniqueName: \"kubernetes.io/projected/73b3c213-f669-469c-aa17-aab28f9fc2c2-kube-api-access-k9m7t\") pod \"payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs\" (UID: \"73b3c213-f669-469c-aa17-aab28f9fc2c2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:22.342269 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.342244    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs7sd\" (UniqueName: \"kubernetes.io/projected/e85a3441-f45b-47a6-acd0-aea38b2df7bf-kube-api-access-hs7sd\") pod \"payload-processing-e2e-api-a-dc3099-69445df59-md827\" (UID: \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:22.413370 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.413329    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:22.437913 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.437880    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:22.688838 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.688805    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"]
Sep 09 21:14:22.690041 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:14:22.690020    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b3c213_f669_469c_aa17_aab28f9fc2c2.slice/crio-881be19c79ef92c0ebc03352445892d99660301fb58764ffd7cc9f769487c2ce WatchSource:0}: Error finding container 881be19c79ef92c0ebc03352445892d99660301fb58764ffd7cc9f769487c2ce: Status 404 returned error can't find the container with id 881be19c79ef92c0ebc03352445892d99660301fb58764ffd7cc9f769487c2ce
Sep 09 21:14:22.724708 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:22.724630    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"]
Sep 09 21:14:22.727306 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:14:22.727275    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85a3441_f45b_47a6_acd0_aea38b2df7bf.slice/crio-59e0b9cd36286d94c6894841f8668c372cae8f250d24c82a9db21e2b224bf31e WatchSource:0}: Error finding container 59e0b9cd36286d94c6894841f8668c372cae8f250d24c82a9db21e2b224bf31e: Status 404 returned error can't find the container with id 59e0b9cd36286d94c6894841f8668c372cae8f250d24c82a9db21e2b224bf31e
Sep 09 21:14:23.315417 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:23.315376    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827" event={"ID":"e85a3441-f45b-47a6-acd0-aea38b2df7bf","Type":"ContainerStarted","Data":"35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda"}
Sep 09 21:14:23.315656 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:23.315436    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827" event={"ID":"e85a3441-f45b-47a6-acd0-aea38b2df7bf","Type":"ContainerStarted","Data":"59e0b9cd36286d94c6894841f8668c372cae8f250d24c82a9db21e2b224bf31e"}
Sep 09 21:14:23.315656 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:23.315565    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:23.317227 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:23.317204    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs" event={"ID":"73b3c213-f669-469c-aa17-aab28f9fc2c2","Type":"ContainerStarted","Data":"238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6"}
Sep 09 21:14:23.317361 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:23.317238    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs" event={"ID":"73b3c213-f669-469c-aa17-aab28f9fc2c2","Type":"ContainerStarted","Data":"881be19c79ef92c0ebc03352445892d99660301fb58764ffd7cc9f769487c2ce"}
Sep 09 21:14:23.317361 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:23.317314    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:23.333690 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:23.333644    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827" podStartSLOduration=1.333631866 podStartE2EDuration="1.333631866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:23.331445741 +0000 UTC m=+1557.302012462" watchObservedRunningTime="2026-09-09 21:14:23.333631866 +0000 UTC m=+1557.304198576"
Sep 09 21:14:23.350273 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:23.350224    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs" podStartSLOduration=1.350208893 podStartE2EDuration="1.350208893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:23.347809855 +0000 UTC m=+1557.318376573" watchObservedRunningTime="2026-09-09 21:14:23.350208893 +0000 UTC m=+1557.320775653"
Sep 09 21:14:26.833885 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:26.833852    2571 scope.go:117] "RemoveContainer" containerID="9a959993ec93388cc1774b1ede42a1995bdb0081d3b51b3addbfb617ffce8b3d"
Sep 09 21:14:34.325402 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:34.325370    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:14:34.325863 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:34.325452    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:14:45.141971 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.141933    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:45.159023 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.158995    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7604-6e130c"]
Sep 09 21:14:45.164345 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.164320    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:45.166788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.166768    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cmlrc\""
Sep 09 21:14:45.167775 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.167749    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-6e130c"]
Sep 09 21:14:45.233025 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.232992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwcjv\" (UniqueName: \"kubernetes.io/projected/4250c4bb-1082-4d13-a815-0700850d504d-kube-api-access-cwcjv\") pod \"test-curl-7604-6e130c\" (UID: \"4250c4bb-1082-4d13-a815-0700850d504d\") " pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:45.333761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.333731    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwcjv\" (UniqueName: \"kubernetes.io/projected/4250c4bb-1082-4d13-a815-0700850d504d-kube-api-access-cwcjv\") pod \"test-curl-7604-6e130c\" (UID: \"4250c4bb-1082-4d13-a815-0700850d504d\") " pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:45.341922 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.341891    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwcjv\" (UniqueName: \"kubernetes.io/projected/4250c4bb-1082-4d13-a815-0700850d504d-kube-api-access-cwcjv\") pod \"test-curl-7604-6e130c\" (UID: \"4250c4bb-1082-4d13-a815-0700850d504d\") " pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:45.476189 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.476155    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:45.571740 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.571701    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:45.750441 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.750417    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-6e130c"]
Sep 09 21:14:45.757306 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:14:45.753616    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4250c4bb_1082_4d13_a815_0700850d504d.slice/crio-530c8ab24a1c80d83d0cd6b82464d49ffe71f8750c9f9199c85f3dbbd8f5bec3 WatchSource:0}: Error finding container 530c8ab24a1c80d83d0cd6b82464d49ffe71f8750c9f9199c85f3dbbd8f5bec3: Status 404 returned error can't find the container with id 530c8ab24a1c80d83d0cd6b82464d49ffe71f8750c9f9199c85f3dbbd8f5bec3
Sep 09 21:14:45.920263 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:45.920226    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:46.461519 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:46.461479    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-6e130c" event={"ID":"4250c4bb-1082-4d13-a815-0700850d504d","Type":"ContainerStarted","Data":"f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63"}
Sep 09 21:14:46.461898 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:46.461534    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-6e130c" event={"ID":"4250c4bb-1082-4d13-a815-0700850d504d","Type":"ContainerStarted","Data":"530c8ab24a1c80d83d0cd6b82464d49ffe71f8750c9f9199c85f3dbbd8f5bec3"}
Sep 09 21:14:46.480827 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:46.480777    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7604-6e130c" podStartSLOduration=1.48076226 podStartE2EDuration="1.48076226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:46.47825465 +0000 UTC m=+1580.448821369" watchObservedRunningTime="2026-09-09 21:14:46.48076226 +0000 UTC m=+1580.451328976"
Sep 09 21:14:46.659887 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:46.659845    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:47.053501 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.053469    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7604-6e130c"]
Sep 09 21:14:47.059822 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.059794    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7604-6e130c"]
Sep 09 21:14:47.171338 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.171251    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7604-52b3fd"]
Sep 09 21:14:47.171826 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.171808    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4250c4bb-1082-4d13-a815-0700850d504d" containerName="test-curl-7604-6e130c"
Sep 09 21:14:47.171885 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.171828    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4250c4bb-1082-4d13-a815-0700850d504d" containerName="test-curl-7604-6e130c"
Sep 09 21:14:47.171928 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.171918    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4250c4bb-1082-4d13-a815-0700850d504d" containerName="test-curl-7604-6e130c"
Sep 09 21:14:47.175284 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.175270    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:47.186563 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.186533    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-52b3fd"]
Sep 09 21:14:47.249904 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.249870    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs7r9\" (UniqueName: \"kubernetes.io/projected/f92819ad-0927-4152-a244-65be6d601826-kube-api-access-bs7r9\") pod \"test-curl-7604-52b3fd\" (UID: \"f92819ad-0927-4152-a244-65be6d601826\") " pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:47.351162 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.351129    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs7r9\" (UniqueName: \"kubernetes.io/projected/f92819ad-0927-4152-a244-65be6d601826-kube-api-access-bs7r9\") pod \"test-curl-7604-52b3fd\" (UID: \"f92819ad-0927-4152-a244-65be6d601826\") " pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:47.361126 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.361068    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs7r9\" (UniqueName: \"kubernetes.io/projected/f92819ad-0927-4152-a244-65be6d601826-kube-api-access-bs7r9\") pod \"test-curl-7604-52b3fd\" (UID: \"f92819ad-0927-4152-a244-65be6d601826\") " pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:47.469144 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:47.469106    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7604-6e130c" container="test-curl-7604-6e130c"
Sep 09 21:14:47.485246 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.485219    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:47.757302 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:47.757276    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7604-52b3fd"]
Sep 09 21:14:47.758900 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:14:47.758879    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf92819ad_0927_4152_a244_65be6d601826.slice/crio-1aafb9dc47e8f8998b483431368d546b9b2abd81cda4ff5d9407ed30f2a7d775 WatchSource:0}: Error finding container 1aafb9dc47e8f8998b483431368d546b9b2abd81cda4ff5d9407ed30f2a7d775: Status 404 returned error can't find the container with id 1aafb9dc47e8f8998b483431368d546b9b2abd81cda4ff5d9407ed30f2a7d775
Sep 09 21:14:48.477445 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:48.477412    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-52b3fd" event={"ID":"f92819ad-0927-4152-a244-65be6d601826","Type":"ContainerStarted","Data":"12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211"}
Sep 09 21:14:48.477920 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:48.477457    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7604-52b3fd" event={"ID":"f92819ad-0927-4152-a244-65be6d601826","Type":"ContainerStarted","Data":"1aafb9dc47e8f8998b483431368d546b9b2abd81cda4ff5d9407ed30f2a7d775"}
Sep 09 21:14:48.477920 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:48.477509    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7604-6e130c" podUID="4250c4bb-1082-4d13-a815-0700850d504d" containerName="test-curl-7604-6e130c" containerID="cri-o://f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63" gracePeriod=2
Sep 09 21:14:48.479780 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:48.479753    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-6e130c\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="4250c4bb-1082-4d13-a815-0700850d504d" pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:48.497885 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:48.497855    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-6e130c\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="4250c4bb-1082-4d13-a815-0700850d504d" pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:48.499188 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:48.499141    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7604-52b3fd" podStartSLOduration=1.499126483 podStartE2EDuration="1.499126483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:48.495796223 +0000 UTC m=+1582.466362940" watchObservedRunningTime="2026-09-09 21:14:48.499126483 +0000 UTC m=+1582.469693200"
Sep 09 21:14:49.104158 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:49.104124    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7604-52b3fd"]
Sep 09 21:14:49.111640 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:49.111607    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7604-52b3fd"]
Sep 09 21:14:49.332611 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:49.332576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:49.480998 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:49.480959    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:49.484258 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:49.484234    2571 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7604-52b3fd" container="test-curl-7604-52b3fd"
Sep 09 21:14:49.583703 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:49.583606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:50.490652 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:50.490621    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7604-52b3fd" podUID="f92819ad-0927-4152-a244-65be6d601826" containerName="test-curl-7604-52b3fd" containerID="cri-o://12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211" gracePeriod=2
Sep 09 21:14:50.492804 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:50.492776    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-52b3fd\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="f92819ad-0927-4152-a244-65be6d601826" pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:50.589835 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:50.589794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:50.677784 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:50.677751    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:50.691834 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:50.691799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:50.785476 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:50.785383    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwcjv\" (UniqueName: \"kubernetes.io/projected/4250c4bb-1082-4d13-a815-0700850d504d-kube-api-access-cwcjv\") pod \"4250c4bb-1082-4d13-a815-0700850d504d\" (UID: \"4250c4bb-1082-4d13-a815-0700850d504d\") "
Sep 09 21:14:50.787727 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:50.787690    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4250c4bb-1082-4d13-a815-0700850d504d-kube-api-access-cwcjv" (OuterVolumeSpecName: "kube-api-access-cwcjv") pod "4250c4bb-1082-4d13-a815-0700850d504d" (UID: "4250c4bb-1082-4d13-a815-0700850d504d"). InnerVolumeSpecName "kube-api-access-cwcjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:14:50.886788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:50.886756    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwcjv\" (UniqueName: \"kubernetes.io/projected/4250c4bb-1082-4d13-a815-0700850d504d-kube-api-access-cwcjv\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:14:51.119071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.118979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:51.230540 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.230501    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:51.429179 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.429102    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"]
Sep 09 21:14:51.429599 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.429586    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f92819ad-0927-4152-a244-65be6d601826" containerName="test-curl-7604-52b3fd"
Sep 09 21:14:51.429660 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.429602    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f92819ad-0927-4152-a244-65be6d601826" containerName="test-curl-7604-52b3fd"
Sep 09 21:14:51.429703 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.429693    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f92819ad-0927-4152-a244-65be6d601826" containerName="test-curl-7604-52b3fd"
Sep 09 21:14:51.433050 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.433033    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:51.438263 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.438243    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-bd9741-dockercfg-2qbrm\""
Sep 09 21:14:51.441263 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.441245    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-bd9741\""
Sep 09 21:14:51.450012 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.449986    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"]
Sep 09 21:14:51.492885 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.492853    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2jcj\" (UniqueName: \"kubernetes.io/projected/71066f51-b807-46cb-b80a-a20a37ed3bcc-kube-api-access-z2jcj\") pod \"payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t\" (UID: \"71066f51-b807-46cb-b80a-a20a37ed3bcc\") " pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:51.493326 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.492905    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71066f51-b807-46cb-b80a-a20a37ed3bcc-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t\" (UID: \"71066f51-b807-46cb-b80a-a20a37ed3bcc\") " pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:51.498295 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.498267    2571 generic.go:356] "Generic (PLEG): container finished" podID="4250c4bb-1082-4d13-a815-0700850d504d" containerID="f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63" exitCode=137
Sep 09 21:14:51.498477 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.498324    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:51.498477 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.498360    2571 scope.go:117] "RemoveContainer" containerID="f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63"
Sep 09 21:14:51.508803 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:51.508751    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-6e130c\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="4250c4bb-1082-4d13-a815-0700850d504d" pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:51.509233 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.509212    2571 scope.go:117] "RemoveContainer" containerID="f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63"
Sep 09 21:14:51.509562 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:51.509542    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63\": container with ID starting with f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63 not found: ID does not exist" containerID="f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63"
Sep 09 21:14:51.509632 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.509571    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63"} err="failed to get container status \"f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63\": rpc error: code = NotFound desc = could not find container \"f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63\": container with ID starting with f39ccd56089fb034a3d6651d353da03068bf1afbb8165191f486ceed098bba63 not found: ID does not exist"
Sep 09 21:14:51.594209 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.594177    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2jcj\" (UniqueName: \"kubernetes.io/projected/71066f51-b807-46cb-b80a-a20a37ed3bcc-kube-api-access-z2jcj\") pod \"payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t\" (UID: \"71066f51-b807-46cb-b80a-a20a37ed3bcc\") " pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:51.594391 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.594234    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71066f51-b807-46cb-b80a-a20a37ed3bcc-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t\" (UID: \"71066f51-b807-46cb-b80a-a20a37ed3bcc\") " pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:51.594938 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.594918    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71066f51-b807-46cb-b80a-a20a37ed3bcc-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t\" (UID: \"71066f51-b807-46cb-b80a-a20a37ed3bcc\") " pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:51.603140 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.603112    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2jcj\" (UniqueName: \"kubernetes.io/projected/71066f51-b807-46cb-b80a-a20a37ed3bcc-kube-api-access-z2jcj\") pod \"payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t\" (UID: \"71066f51-b807-46cb-b80a-a20a37ed3bcc\") " pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:51.748262 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:51.748220    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:52.027714 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.027687    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"]
Sep 09 21:14:52.029171 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:14:52.029134    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71066f51_b807_46cb_b80a_a20a37ed3bcc.slice/crio-7a5a870bbfeebd09ac6fbfcc87e5c70533a84dd568abfb7d1f37d215e3444bce WatchSource:0}: Error finding container 7a5a870bbfeebd09ac6fbfcc87e5c70533a84dd568abfb7d1f37d215e3444bce: Status 404 returned error can't find the container with id 7a5a870bbfeebd09ac6fbfcc87e5c70533a84dd568abfb7d1f37d215e3444bce
Sep 09 21:14:52.506528 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.506496    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t" event={"ID":"71066f51-b807-46cb-b80a-a20a37ed3bcc","Type":"ContainerStarted","Data":"05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac"}
Sep 09 21:14:52.506954 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.506554    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t" event={"ID":"71066f51-b807-46cb-b80a-a20a37ed3bcc","Type":"ContainerStarted","Data":"7a5a870bbfeebd09ac6fbfcc87e5c70533a84dd568abfb7d1f37d215e3444bce"}
Sep 09 21:14:52.506954 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.506579    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:14:52.508468 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:52.508443    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-6e130c\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="4250c4bb-1082-4d13-a815-0700850d504d" pod="opendatahub/test-curl-7604-6e130c"
Sep 09 21:14:52.534763 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.534689    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t" podStartSLOduration=1.5346681009999998 podStartE2EDuration="1.534668101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:14:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:14:52.524622731 +0000 UTC m=+1586.495189449" watchObservedRunningTime="2026-09-09 21:14:52.534668101 +0000 UTC m=+1586.505234824"
Sep 09 21:14:52.567628 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.567598    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4250c4bb-1082-4d13-a815-0700850d504d" path="/var/lib/kubelet/pods/4250c4bb-1082-4d13-a815-0700850d504d/volumes"
Sep 09 21:14:52.695651 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.695622    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:52.713611 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.713583    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:52.806962 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.806873    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs7r9\" (UniqueName: \"kubernetes.io/projected/f92819ad-0927-4152-a244-65be6d601826-kube-api-access-bs7r9\") pod \"f92819ad-0927-4152-a244-65be6d601826\" (UID: \"f92819ad-0927-4152-a244-65be6d601826\") "
Sep 09 21:14:52.809184 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.809161    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f92819ad-0927-4152-a244-65be6d601826-kube-api-access-bs7r9" (OuterVolumeSpecName: "kube-api-access-bs7r9") pod "f92819ad-0927-4152-a244-65be6d601826" (UID: "f92819ad-0927-4152-a244-65be6d601826"). InnerVolumeSpecName "kube-api-access-bs7r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:14:52.907679 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:52.907644    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs7r9\" (UniqueName: \"kubernetes.io/projected/f92819ad-0927-4152-a244-65be6d601826-kube-api-access-bs7r9\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:14:53.515545 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:53.515512    2571 generic.go:356] "Generic (PLEG): container finished" podID="f92819ad-0927-4152-a244-65be6d601826" containerID="12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211" exitCode=137
Sep 09 21:14:53.515991 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:53.515554    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:53.515991 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:53.515617    2571 scope.go:117] "RemoveContainer" containerID="12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211"
Sep 09 21:14:53.525589 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:53.525573    2571 scope.go:117] "RemoveContainer" containerID="12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211"
Sep 09 21:14:53.525741 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:53.525717    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-52b3fd\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="f92819ad-0927-4152-a244-65be6d601826" pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:53.525888 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:53.525868    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211\": container with ID starting with 12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211 not found: ID does not exist" containerID="12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211"
Sep 09 21:14:53.525934 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:53.525898    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211"} err="failed to get container status \"12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211\": rpc error: code = NotFound desc = could not find container \"12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211\": container with ID starting with 12d7a4d834c22b4e63d7624e3238e3336be9ed67140a681626a6107d1ba32211 not found: ID does not exist"
Sep 09 21:14:53.990492 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:53.990457    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"]
Sep 09 21:14:53.993061 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:14:53.993015    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7604-52b3fd\" is forbidden: User \"system:node:ip-10-0-129-13.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-13.ec2.internal' and this object" podUID="f92819ad-0927-4152-a244-65be6d601826" pod="opendatahub/test-curl-7604-52b3fd"
Sep 09 21:14:54.001636 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:54.001608    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:14:54.566665 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:14:54.566630    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f92819ad-0927-4152-a244-65be6d601826" path="/var/lib/kubelet/pods/f92819ad-0927-4152-a244-65be6d601826/volumes"
Sep 09 21:15:00.148430 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.148393    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8"]
Sep 09 21:15:00.152941 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.152917    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"]
Sep 09 21:15:00.153123 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.153032    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8"
Sep 09 21:15:00.156162 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.156132    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:15:00.157454 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.157434    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"]
Sep 09 21:15:00.157578 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.157558    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"
Sep 09 21:15:00.161237 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.161220    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"]
Sep 09 21:15:00.161358 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.161346    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"
Sep 09 21:15:00.165057 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.165035    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"
Sep 09 21:15:00.167236 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.167216    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8"
Sep 09 21:15:00.167822 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.167799    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8"]
Sep 09 21:15:00.173311 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.173288    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"]
Sep 09 21:15:00.178526 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.177784    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"
Sep 09 21:15:00.178526 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.177991    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"]
Sep 09 21:15:00.181825 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.181800    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"]
Sep 09 21:15:00.191326 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.191301    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"]
Sep 09 21:15:00.197893 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.197880    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"
Sep 09 21:15:00.203730 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.203626    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"]
Sep 09 21:15:00.221584 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.221559    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"
Sep 09 21:15:00.235668 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.235586    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"
Sep 09 21:15:00.291995 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.291796    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"
Sep 09 21:15:00.474347 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.474316    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"]
Sep 09 21:15:00.475874 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:15:00.475853    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6780cc41_21a3_4581_b2f5_8f252842a3f3.slice/crio-ef766a97e25b1e9df7407d744fcc2cc510af24bcd5169dbd96a2a7ee2429c65e WatchSource:0}: Error finding container ef766a97e25b1e9df7407d744fcc2cc510af24bcd5169dbd96a2a7ee2429c65e: Status 404 returned error can't find the container with id ef766a97e25b1e9df7407d744fcc2cc510af24bcd5169dbd96a2a7ee2429c65e
Sep 09 21:15:00.480323 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.480302    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8"]
Sep 09 21:15:00.481073 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:15:00.481051    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4ffccb_dfe7_471c_b170_ad3cd61b5e21.slice/crio-d828eb4fa4e0051ca2502ea5b7dacc27026276b3ba86fe2207aafdb2bf1395cf WatchSource:0}: Error finding container d828eb4fa4e0051ca2502ea5b7dacc27026276b3ba86fe2207aafdb2bf1395cf: Status 404 returned error can't find the container with id d828eb4fa4e0051ca2502ea5b7dacc27026276b3ba86fe2207aafdb2bf1395cf
Sep 09 21:15:00.522170 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.522143    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"]
Sep 09 21:15:00.523551 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:15:00.523527    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa47000_0034_485d_8709_7e9813b42d81.slice/crio-c3febe0e7962716980b60b9ba38616bc8312482de7cfc34b307756cc46514f22 WatchSource:0}: Error finding container c3febe0e7962716980b60b9ba38616bc8312482de7cfc34b307756cc46514f22: Status 404 returned error can't find the container with id c3febe0e7962716980b60b9ba38616bc8312482de7cfc34b307756cc46514f22
Sep 09 21:15:00.542893 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.542870    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"]
Sep 09 21:15:00.545318 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:15:00.545212    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ae43fb_b3e1_4950_ab67_cee65ef4770d.slice/crio-93f4f2ad24ac295d8e018dca42d9485ef59a8314d575d1dae68d51e2554a2fd1 WatchSource:0}: Error finding container 93f4f2ad24ac295d8e018dca42d9485ef59a8314d575d1dae68d51e2554a2fd1: Status 404 returned error can't find the container with id 93f4f2ad24ac295d8e018dca42d9485ef59a8314d575d1dae68d51e2554a2fd1
Sep 09 21:15:00.569592 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.569560    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn" event={"ID":"6780cc41-21a3-4581-b2f5-8f252842a3f3","Type":"ContainerStarted","Data":"ef766a97e25b1e9df7407d744fcc2cc510af24bcd5169dbd96a2a7ee2429c65e"}
Sep 09 21:15:00.569701 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.569619    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8" event={"ID":"2a4ffccb-dfe7-471c-b170-ad3cd61b5e21","Type":"ContainerStarted","Data":"d828eb4fa4e0051ca2502ea5b7dacc27026276b3ba86fe2207aafdb2bf1395cf"}
Sep 09 21:15:00.570287 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.570257    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq" event={"ID":"45ae43fb-b3e1-4950-ab67-cee65ef4770d","Type":"ContainerStarted","Data":"93f4f2ad24ac295d8e018dca42d9485ef59a8314d575d1dae68d51e2554a2fd1"}
Sep 09 21:15:00.571529 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.571511    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb" event={"ID":"5fa47000-0034-485d-8709-7e9813b42d81","Type":"ContainerStarted","Data":"c3febe0e7962716980b60b9ba38616bc8312482de7cfc34b307756cc46514f22"}
Sep 09 21:15:00.582432 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:00.582414    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"]
Sep 09 21:15:00.584027 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:15:00.584005    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ec01e37_859b_4f37_b3fa_e4640985a0c5.slice/crio-1284099f7c2f224a467fbaa6ab3ee2e4976898fdc09cb90e3ab22e50515eaf8b WatchSource:0}: Error finding container 1284099f7c2f224a467fbaa6ab3ee2e4976898fdc09cb90e3ab22e50515eaf8b: Status 404 returned error can't find the container with id 1284099f7c2f224a467fbaa6ab3ee2e4976898fdc09cb90e3ab22e50515eaf8b
Sep 09 21:15:01.579349 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.579320    2571 generic.go:356] "Generic (PLEG): container finished" podID="2a4ffccb-dfe7-471c-b170-ad3cd61b5e21" containerID="9df9b0896db05c68c21df948bd0c10ff9d9ed71a8c5df5c8e75291538927bfaf" exitCode=0
Sep 09 21:15:01.579670 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.579401    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8" event={"ID":"2a4ffccb-dfe7-471c-b170-ad3cd61b5e21","Type":"ContainerDied","Data":"9df9b0896db05c68c21df948bd0c10ff9d9ed71a8c5df5c8e75291538927bfaf"}
Sep 09 21:15:01.581059 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.581035    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq" event={"ID":"45ae43fb-b3e1-4950-ab67-cee65ef4770d","Type":"ContainerStarted","Data":"5aa6a98a1016d73d0bb1aeeb1715b2560cc474ce6e7a35aeb0f4533463438b23"}
Sep 09 21:15:01.582787 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.582763    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb" event={"ID":"5fa47000-0034-485d-8709-7e9813b42d81","Type":"ContainerStarted","Data":"b9c40f471b0b38bcb55a8b12085c9a98381c6e0cf77b61282e99341f2019a2ab"}
Sep 09 21:15:01.584227 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.584206    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn" event={"ID":"6780cc41-21a3-4581-b2f5-8f252842a3f3","Type":"ContainerStarted","Data":"e7ac582a164252f02365372da5dc9f34df0b8d07e9c65b1b4df87fd3d1bc7aed"}
Sep 09 21:15:01.585711 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.585693    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw" event={"ID":"5ec01e37-859b-4f37-b3fa-e4640985a0c5","Type":"ContainerStarted","Data":"00a4f46292931e480341c1339da5c3f64c6e3c4fc534b08d27b5eb8412799d57"}
Sep 09 21:15:01.585787 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.585732    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw" event={"ID":"5ec01e37-859b-4f37-b3fa-e4640985a0c5","Type":"ContainerStarted","Data":"1284099f7c2f224a467fbaa6ab3ee2e4976898fdc09cb90e3ab22e50515eaf8b"}
Sep 09 21:15:01.609114 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.609055    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw" podStartSLOduration=1.6090366839999999 podStartE2EDuration="1.609036684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:01.607300453 +0000 UTC m=+1595.577867170" watchObservedRunningTime="2026-09-09 21:15:01.609036684 +0000 UTC m=+1595.579603465"
Sep 09 21:15:01.626409 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.626345    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb" podStartSLOduration=1.626323238 podStartE2EDuration="1.626323238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:01.623036709 +0000 UTC m=+1595.593603425" watchObservedRunningTime="2026-09-09 21:15:01.626323238 +0000 UTC m=+1595.596889958"
Sep 09 21:15:01.639764 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.639717    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn" podStartSLOduration=1.639702256 podStartE2EDuration="1.639702256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:01.638911451 +0000 UTC m=+1595.609478172" watchObservedRunningTime="2026-09-09 21:15:01.639702256 +0000 UTC m=+1595.610268972"
Sep 09 21:15:01.656639 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.656575    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq" podStartSLOduration=1.656554688 podStartE2EDuration="1.656554688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:15:01.654555405 +0000 UTC m=+1595.625122124" watchObservedRunningTime="2026-09-09 21:15:01.656554688 +0000 UTC m=+1595.627121408"
Sep 09 21:15:01.952181 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:01.952142    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:02.593946 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.593915    2571 generic.go:356] "Generic (PLEG): container finished" podID="45ae43fb-b3e1-4950-ab67-cee65ef4770d" containerID="5aa6a98a1016d73d0bb1aeeb1715b2560cc474ce6e7a35aeb0f4533463438b23" exitCode=0
Sep 09 21:15:02.594409 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.594005    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq" event={"ID":"45ae43fb-b3e1-4950-ab67-cee65ef4770d","Type":"ContainerDied","Data":"5aa6a98a1016d73d0bb1aeeb1715b2560cc474ce6e7a35aeb0f4533463438b23"}
Sep 09 21:15:02.595593 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.595567    2571 generic.go:356] "Generic (PLEG): container finished" podID="5fa47000-0034-485d-8709-7e9813b42d81" containerID="b9c40f471b0b38bcb55a8b12085c9a98381c6e0cf77b61282e99341f2019a2ab" exitCode=0
Sep 09 21:15:02.595688 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.595648    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb" event={"ID":"5fa47000-0034-485d-8709-7e9813b42d81","Type":"ContainerDied","Data":"b9c40f471b0b38bcb55a8b12085c9a98381c6e0cf77b61282e99341f2019a2ab"}
Sep 09 21:15:02.597148 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.597126    2571 generic.go:356] "Generic (PLEG): container finished" podID="6780cc41-21a3-4581-b2f5-8f252842a3f3" containerID="e7ac582a164252f02365372da5dc9f34df0b8d07e9c65b1b4df87fd3d1bc7aed" exitCode=0
Sep 09 21:15:02.597242 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.597193    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn" event={"ID":"6780cc41-21a3-4581-b2f5-8f252842a3f3","Type":"ContainerDied","Data":"e7ac582a164252f02365372da5dc9f34df0b8d07e9c65b1b4df87fd3d1bc7aed"}
Sep 09 21:15:02.598755 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.598736    2571 generic.go:356] "Generic (PLEG): container finished" podID="5ec01e37-859b-4f37-b3fa-e4640985a0c5" containerID="00a4f46292931e480341c1339da5c3f64c6e3c4fc534b08d27b5eb8412799d57" exitCode=0
Sep 09 21:15:02.598847 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.598812    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw" event={"ID":"5ec01e37-859b-4f37-b3fa-e4640985a0c5","Type":"ContainerDied","Data":"00a4f46292931e480341c1339da5c3f64c6e3c4fc534b08d27b5eb8412799d57"}
Sep 09 21:15:02.752588 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:02.752564    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8"
Sep 09 21:15:03.518182 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:03.518152    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:15:03.607231 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:03.607114    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8" event={"ID":"2a4ffccb-dfe7-471c-b170-ad3cd61b5e21","Type":"ContainerDied","Data":"d828eb4fa4e0051ca2502ea5b7dacc27026276b3ba86fe2207aafdb2bf1395cf"}
Sep 09 21:15:03.607231 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:03.607137    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29816475-zqnx8"
Sep 09 21:15:03.607231 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:03.607180    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d828eb4fa4e0051ca2502ea5b7dacc27026276b3ba86fe2207aafdb2bf1395cf"
Sep 09 21:15:03.763787 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:03.763731    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"
Sep 09 21:15:03.809712 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:03.809684    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"
Sep 09 21:15:03.846071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:03.846044    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"
Sep 09 21:15:03.849409 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:03.849388    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"
Sep 09 21:15:04.510234 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.510184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:04.617217 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.617194    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"
Sep 09 21:15:04.617575 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.617228    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq" event={"ID":"45ae43fb-b3e1-4950-ab67-cee65ef4770d","Type":"ContainerDied","Data":"93f4f2ad24ac295d8e018dca42d9485ef59a8314d575d1dae68d51e2554a2fd1"}
Sep 09 21:15:04.617575 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.617287    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93f4f2ad24ac295d8e018dca42d9485ef59a8314d575d1dae68d51e2554a2fd1"
Sep 09 21:15:04.618748 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.618726    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"
Sep 09 21:15:04.618867 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.618758    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb" event={"ID":"5fa47000-0034-485d-8709-7e9813b42d81","Type":"ContainerDied","Data":"c3febe0e7962716980b60b9ba38616bc8312482de7cfc34b307756cc46514f22"}
Sep 09 21:15:04.618867 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.618799    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3febe0e7962716980b60b9ba38616bc8312482de7cfc34b307756cc46514f22"
Sep 09 21:15:04.620558 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.620539    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"
Sep 09 21:15:04.620558 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.620553    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn" event={"ID":"6780cc41-21a3-4581-b2f5-8f252842a3f3","Type":"ContainerDied","Data":"ef766a97e25b1e9df7407d744fcc2cc510af24bcd5169dbd96a2a7ee2429c65e"}
Sep 09 21:15:04.620728 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.620587    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef766a97e25b1e9df7407d744fcc2cc510af24bcd5169dbd96a2a7ee2429c65e"
Sep 09 21:15:04.622028 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.622012    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"
Sep 09 21:15:04.622168 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.622061    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw" event={"ID":"5ec01e37-859b-4f37-b3fa-e4640985a0c5","Type":"ContainerDied","Data":"1284099f7c2f224a467fbaa6ab3ee2e4976898fdc09cb90e3ab22e50515eaf8b"}
Sep 09 21:15:04.622168 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:04.622130    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1284099f7c2f224a467fbaa6ab3ee2e4976898fdc09cb90e3ab22e50515eaf8b"
Sep 09 21:15:09.011743 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.011705    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj" podUID="d0bcfba8-d621-42a9-a40d-158cefa399bc" containerName="main" containerID="cri-o://f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038" gracePeriod=45
Sep 09 21:15:09.265557 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.265498    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:15:09.379143 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379067    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-home\") pod \"d0bcfba8-d621-42a9-a40d-158cefa399bc\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") "
Sep 09 21:15:09.379323 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379176    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjh66\" (UniqueName: \"kubernetes.io/projected/d0bcfba8-d621-42a9-a40d-158cefa399bc-kube-api-access-kjh66\") pod \"d0bcfba8-d621-42a9-a40d-158cefa399bc\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") "
Sep 09 21:15:09.379323 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379199    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0bcfba8-d621-42a9-a40d-158cefa399bc-tls-certs\") pod \"d0bcfba8-d621-42a9-a40d-158cefa399bc\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") "
Sep 09 21:15:09.379323 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379274    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-model-cache\") pod \"d0bcfba8-d621-42a9-a40d-158cefa399bc\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") "
Sep 09 21:15:09.379323 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379313    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-tmp-dir\") pod \"d0bcfba8-d621-42a9-a40d-158cefa399bc\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") "
Sep 09 21:15:09.379484 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379333    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-dshm\") pod \"d0bcfba8-d621-42a9-a40d-158cefa399bc\" (UID: \"d0bcfba8-d621-42a9-a40d-158cefa399bc\") "
Sep 09 21:15:09.379484 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379351    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-home" (OuterVolumeSpecName: "home") pod "d0bcfba8-d621-42a9-a40d-158cefa399bc" (UID: "d0bcfba8-d621-42a9-a40d-158cefa399bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:15:09.379611 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379571    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0bcfba8-d621-42a9-a40d-158cefa399bc" (UID: "d0bcfba8-d621-42a9-a40d-158cefa399bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:15:09.379611 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379586    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0bcfba8-d621-42a9-a40d-158cefa399bc" (UID: "d0bcfba8-d621-42a9-a40d-158cefa399bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:15:09.379689 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379649    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-model-cache\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:09.379689 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379661    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-tmp-dir\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:09.379689 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.379674    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-home\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:09.381788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.381760    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0bcfba8-d621-42a9-a40d-158cefa399bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0bcfba8-d621-42a9-a40d-158cefa399bc" (UID: "d0bcfba8-d621-42a9-a40d-158cefa399bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:15:09.381918 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.381808    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-dshm" (OuterVolumeSpecName: "dshm") pod "d0bcfba8-d621-42a9-a40d-158cefa399bc" (UID: "d0bcfba8-d621-42a9-a40d-158cefa399bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:15:09.381918 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.381829    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0bcfba8-d621-42a9-a40d-158cefa399bc-kube-api-access-kjh66" (OuterVolumeSpecName: "kube-api-access-kjh66") pod "d0bcfba8-d621-42a9-a40d-158cefa399bc" (UID: "d0bcfba8-d621-42a9-a40d-158cefa399bc"). InnerVolumeSpecName "kube-api-access-kjh66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:15:09.480966 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.480930    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjh66\" (UniqueName: \"kubernetes.io/projected/d0bcfba8-d621-42a9-a40d-158cefa399bc-kube-api-access-kjh66\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:09.480966 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.480963    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0bcfba8-d621-42a9-a40d-158cefa399bc-tls-certs\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:09.480966 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.480976    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0bcfba8-d621-42a9-a40d-158cefa399bc-dshm\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:09.656910 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.656823    2571 generic.go:356] "Generic (PLEG): container finished" podID="d0bcfba8-d621-42a9-a40d-158cefa399bc" containerID="f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038" exitCode=0
Sep 09 21:15:09.656910 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.656879    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"
Sep 09 21:15:09.656910 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.656900    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj" event={"ID":"d0bcfba8-d621-42a9-a40d-158cefa399bc","Type":"ContainerDied","Data":"f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038"}
Sep 09 21:15:09.657203 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.656942    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj" event={"ID":"d0bcfba8-d621-42a9-a40d-158cefa399bc","Type":"ContainerDied","Data":"47859715e93587d417bc832ee33f5be64709e4bf4f7da416500fe5edcc84a651"}
Sep 09 21:15:09.657203 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.656959    2571 scope.go:117] "RemoveContainer" containerID="f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038"
Sep 09 21:15:09.667690 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.667668    2571 scope.go:117] "RemoveContainer" containerID="f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038"
Sep 09 21:15:09.667990 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:15:09.667971    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038\": container with ID starting with f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038 not found: ID does not exist" containerID="f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038"
Sep 09 21:15:09.668178 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.667999    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038"} err="failed to get container status \"f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038\": rpc error: code = NotFound desc = could not find container \"f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038\": container with ID starting with f26e8be4f5c6de24780e7fbc32ee27084a6dc04f659340848b72e3f5134a1038 not found: ID does not exist"
Sep 09 21:15:09.680629 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.680602    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"]
Sep 09 21:15:09.699072 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:09.699043    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8db5a7/sub-test-model-8db5a7-kserve-67d955cb65-27bsj"]
Sep 09 21:15:10.566827 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:10.566785    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0bcfba8-d621-42a9-a40d-158cefa399bc" path="/var/lib/kubelet/pods/d0bcfba8-d621-42a9-a40d-158cefa399bc/volumes"
Sep 09 21:15:11.747969 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:11.747937    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:14.337592 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:14.337561    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:17.255891 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.255861    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"]
Sep 09 21:15:17.256468 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256450    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6780cc41-21a3-4581-b2f5-8f252842a3f3" containerName="cleanup"
Sep 09 21:15:17.256514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256471    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6780cc41-21a3-4581-b2f5-8f252842a3f3" containerName="cleanup"
Sep 09 21:15:17.256514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256495    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fa47000-0034-485d-8709-7e9813b42d81" containerName="cleanup"
Sep 09 21:15:17.256514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256500    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fa47000-0034-485d-8709-7e9813b42d81" containerName="cleanup"
Sep 09 21:15:17.256514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256512    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45ae43fb-b3e1-4950-ab67-cee65ef4770d" containerName="cleanup"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256517    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="45ae43fb-b3e1-4950-ab67-cee65ef4770d" containerName="cleanup"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256527    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0bcfba8-d621-42a9-a40d-158cefa399bc" containerName="main"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256533    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0bcfba8-d621-42a9-a40d-158cefa399bc" containerName="main"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256543    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a4ffccb-dfe7-471c-b170-ad3cd61b5e21" containerName="cleanup"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256548    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a4ffccb-dfe7-471c-b170-ad3cd61b5e21" containerName="cleanup"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256557    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ec01e37-859b-4f37-b3fa-e4640985a0c5" containerName="cleanup"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256561    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ec01e37-859b-4f37-b3fa-e4640985a0c5" containerName="cleanup"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256639    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="45ae43fb-b3e1-4950-ab67-cee65ef4770d" containerName="cleanup"
Sep 09 21:15:17.256647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256650    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ec01e37-859b-4f37-b3fa-e4640985a0c5" containerName="cleanup"
Sep 09 21:15:17.256917 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256658    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6780cc41-21a3-4581-b2f5-8f252842a3f3" containerName="cleanup"
Sep 09 21:15:17.256917 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256664    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a4ffccb-dfe7-471c-b170-ad3cd61b5e21" containerName="cleanup"
Sep 09 21:15:17.256917 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256673    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0bcfba8-d621-42a9-a40d-158cefa399bc" containerName="main"
Sep 09 21:15:17.256917 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.256678    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fa47000-0034-485d-8709-7e9813b42d81" containerName="cleanup"
Sep 09 21:15:17.261470 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.261452    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"
Sep 09 21:15:17.265892 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.265876    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:15:17.285004 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.284972    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"]
Sep 09 21:15:17.359902 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.359866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da440b47-1ff1-4a5b-95c8-66bf6df40ab4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dc3099-l57tt\" (UID: \"da440b47-1ff1-4a5b-95c8-66bf6df40ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"
Sep 09 21:15:17.393292 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.393262    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:17.460884 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.460854    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da440b47-1ff1-4a5b-95c8-66bf6df40ab4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dc3099-l57tt\" (UID: \"da440b47-1ff1-4a5b-95c8-66bf6df40ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"
Sep 09 21:15:17.461521 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.461500    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da440b47-1ff1-4a5b-95c8-66bf6df40ab4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-dc3099-l57tt\" (UID: \"da440b47-1ff1-4a5b-95c8-66bf6df40ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"
Sep 09 21:15:17.572275 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.572169    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"
Sep 09 21:15:17.852893 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:17.852868    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"]
Sep 09 21:15:17.855178 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:15:17.855154    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda440b47_1ff1_4a5b_95c8_66bf6df40ab4.slice/crio-b0df76425b28c4c47f91affba6cd6ac3eb4d06c8dbab1de6c2b0fe7253e87979 WatchSource:0}: Error finding container b0df76425b28c4c47f91affba6cd6ac3eb4d06c8dbab1de6c2b0fe7253e87979: Status 404 returned error can't find the container with id b0df76425b28c4c47f91affba6cd6ac3eb4d06c8dbab1de6c2b0fe7253e87979
Sep 09 21:15:18.722712 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:18.722681    2571 generic.go:356] "Generic (PLEG): container finished" podID="da440b47-1ff1-4a5b-95c8-66bf6df40ab4" containerID="3d9138e31273a9942330cc52ad194d9a1925bf4eaac907391d55e6423afbe5ac" exitCode=0
Sep 09 21:15:18.723099 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:18.722745    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt" event={"ID":"da440b47-1ff1-4a5b-95c8-66bf6df40ab4","Type":"ContainerDied","Data":"3d9138e31273a9942330cc52ad194d9a1925bf4eaac907391d55e6423afbe5ac"}
Sep 09 21:15:18.723099 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:18.722792    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt" event={"ID":"da440b47-1ff1-4a5b-95c8-66bf6df40ab4","Type":"ContainerStarted","Data":"b0df76425b28c4c47f91affba6cd6ac3eb4d06c8dbab1de6c2b0fe7253e87979"}
Sep 09 21:15:19.873524 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:19.873495    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"
Sep 09 21:15:19.988725 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:19.988636    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da440b47-1ff1-4a5b-95c8-66bf6df40ab4-maas-api-service-ca\") pod \"da440b47-1ff1-4a5b-95c8-66bf6df40ab4\" (UID: \"da440b47-1ff1-4a5b-95c8-66bf6df40ab4\") "
Sep 09 21:15:19.989046 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:19.989021    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da440b47-1ff1-4a5b-95c8-66bf6df40ab4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "da440b47-1ff1-4a5b-95c8-66bf6df40ab4" (UID: "da440b47-1ff1-4a5b-95c8-66bf6df40ab4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:15:19.989201 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:19.989187    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/da440b47-1ff1-4a5b-95c8-66bf6df40ab4-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:20.737238 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:20.737212    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"
Sep 09 21:15:20.737416 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:20.737246    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt" event={"ID":"da440b47-1ff1-4a5b-95c8-66bf6df40ab4","Type":"ContainerDied","Data":"b0df76425b28c4c47f91affba6cd6ac3eb4d06c8dbab1de6c2b0fe7253e87979"}
Sep 09 21:15:20.737416 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:20.737288    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0df76425b28c4c47f91affba6cd6ac3eb4d06c8dbab1de6c2b0fe7253e87979"
Sep 09 21:15:26.032823 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:26.032781    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:26.866355 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:26.866327    2571 scope.go:117] "RemoveContainer" containerID="4bfff0eb14167526fdcdfd83c1e1d5d06f847a5d510895b640d1a868c9679077"
Sep 09 21:15:31.798056 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:31.798020    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:32.730694 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.730663    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"]
Sep 09 21:15:32.731543 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.731507    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:32.733956 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.733932    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-dc3099-29816475-sx9zn"]
Sep 09 21:15:32.777946 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.777909    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"]
Sep 09 21:15:32.778495 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.778452    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827" podUID="e85a3441-f45b-47a6-acd0-aea38b2df7bf" containerName="payload-processing" containerID="cri-o://35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda" gracePeriod=30
Sep 09 21:15:32.782150 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.782125    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"]
Sep 09 21:15:32.782619 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.782568    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs" podUID="73b3c213-f669-469c-aa17-aab28f9fc2c2" containerName="payload-pre-processing" containerID="cri-o://238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6" gracePeriod=30
Sep 09 21:15:32.805751 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.805712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:32.876590 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.876541    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:32.933344 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.933283    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"]
Sep 09 21:15:32.937809 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.937778    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-dc3099-l57tt"]
Sep 09 21:15:32.988519 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:32.988337    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:33.044118 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.044067    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:33.066486 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.066463    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:15:33.069599 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.069582    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:15:33.119254 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.119217    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:33.122880 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.122860    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e85a3441-f45b-47a6-acd0-aea38b2df7bf-plugins-config-volume\") pod \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\" (UID: \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\") "
Sep 09 21:15:33.122993 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.122909    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9m7t\" (UniqueName: \"kubernetes.io/projected/73b3c213-f669-469c-aa17-aab28f9fc2c2-kube-api-access-k9m7t\") pod \"73b3c213-f669-469c-aa17-aab28f9fc2c2\" (UID: \"73b3c213-f669-469c-aa17-aab28f9fc2c2\") "
Sep 09 21:15:33.122993 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.122984    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs7sd\" (UniqueName: \"kubernetes.io/projected/e85a3441-f45b-47a6-acd0-aea38b2df7bf-kube-api-access-hs7sd\") pod \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\" (UID: \"e85a3441-f45b-47a6-acd0-aea38b2df7bf\") "
Sep 09 21:15:33.123147 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.123130    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b3c213-f669-469c-aa17-aab28f9fc2c2-plugins-config-volume\") pod \"73b3c213-f669-469c-aa17-aab28f9fc2c2\" (UID: \"73b3c213-f669-469c-aa17-aab28f9fc2c2\") "
Sep 09 21:15:33.123485 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.123454    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e85a3441-f45b-47a6-acd0-aea38b2df7bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e85a3441-f45b-47a6-acd0-aea38b2df7bf" (UID: "e85a3441-f45b-47a6-acd0-aea38b2df7bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:15:33.123611 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.123493    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73b3c213-f669-469c-aa17-aab28f9fc2c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "73b3c213-f669-469c-aa17-aab28f9fc2c2" (UID: "73b3c213-f669-469c-aa17-aab28f9fc2c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:15:33.125486 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.125465    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73b3c213-f669-469c-aa17-aab28f9fc2c2-kube-api-access-k9m7t" (OuterVolumeSpecName: "kube-api-access-k9m7t") pod "73b3c213-f669-469c-aa17-aab28f9fc2c2" (UID: "73b3c213-f669-469c-aa17-aab28f9fc2c2"). InnerVolumeSpecName "kube-api-access-k9m7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:15:33.125582 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.125502    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e85a3441-f45b-47a6-acd0-aea38b2df7bf-kube-api-access-hs7sd" (OuterVolumeSpecName: "kube-api-access-hs7sd") pod "e85a3441-f45b-47a6-acd0-aea38b2df7bf" (UID: "e85a3441-f45b-47a6-acd0-aea38b2df7bf"). InnerVolumeSpecName "kube-api-access-hs7sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:15:33.224438 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.224403    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e85a3441-f45b-47a6-acd0-aea38b2df7bf-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:33.224438 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.224430    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9m7t\" (UniqueName: \"kubernetes.io/projected/73b3c213-f669-469c-aa17-aab28f9fc2c2-kube-api-access-k9m7t\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:33.224438 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.224439    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs7sd\" (UniqueName: \"kubernetes.io/projected/e85a3441-f45b-47a6-acd0-aea38b2df7bf-kube-api-access-hs7sd\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:33.224672 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.224450    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b3c213-f669-469c-aa17-aab28f9fc2c2-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:33.308419 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.308332    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:33.832880 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.832843    2571 generic.go:356] "Generic (PLEG): container finished" podID="e85a3441-f45b-47a6-acd0-aea38b2df7bf" containerID="35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda" exitCode=0
Sep 09 21:15:33.833368 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.832895    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"
Sep 09 21:15:33.833368 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.832936    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827" event={"ID":"e85a3441-f45b-47a6-acd0-aea38b2df7bf","Type":"ContainerDied","Data":"35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda"}
Sep 09 21:15:33.833368 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.832990    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827" event={"ID":"e85a3441-f45b-47a6-acd0-aea38b2df7bf","Type":"ContainerDied","Data":"59e0b9cd36286d94c6894841f8668c372cae8f250d24c82a9db21e2b224bf31e"}
Sep 09 21:15:33.833368 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.833016    2571 scope.go:117] "RemoveContainer" containerID="35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda"
Sep 09 21:15:33.834392 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.834324    2571 generic.go:356] "Generic (PLEG): container finished" podID="73b3c213-f669-469c-aa17-aab28f9fc2c2" containerID="238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6" exitCode=0
Sep 09 21:15:33.834392 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.834361    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"
Sep 09 21:15:33.834504 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.834419    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs" event={"ID":"73b3c213-f669-469c-aa17-aab28f9fc2c2","Type":"ContainerDied","Data":"238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6"}
Sep 09 21:15:33.834504 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.834468    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs" event={"ID":"73b3c213-f669-469c-aa17-aab28f9fc2c2","Type":"ContainerDied","Data":"881be19c79ef92c0ebc03352445892d99660301fb58764ffd7cc9f769487c2ce"}
Sep 09 21:15:33.844203 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.844118    2571 scope.go:117] "RemoveContainer" containerID="35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda"
Sep 09 21:15:33.844557 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:15:33.844482    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda\": container with ID starting with 35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda not found: ID does not exist" containerID="35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda"
Sep 09 21:15:33.844653 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.844562    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda"} err="failed to get container status \"35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda\": rpc error: code = NotFound desc = could not find container \"35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda\": container with ID starting with 35cf2ed5bc14bc0790fc1a28196a7963e615116bf3ed53192027fccdbf180cda not found: ID does not exist"
Sep 09 21:15:33.844653 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.844596    2571 scope.go:117] "RemoveContainer" containerID="238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6"
Sep 09 21:15:33.855502 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.855486    2571 scope.go:117] "RemoveContainer" containerID="238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6"
Sep 09 21:15:33.855770 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:15:33.855751    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6\": container with ID starting with 238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6 not found: ID does not exist" containerID="238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6"
Sep 09 21:15:33.855837 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.855780    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6"} err="failed to get container status \"238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6\": rpc error: code = NotFound desc = could not find container \"238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6\": container with ID starting with 238a3f0b3b039c32db9cc990da9a2220f0c1ade73a9fd7a8dc4ba62f5efe75f6 not found: ID does not exist"
Sep 09 21:15:33.859525 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.859506    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"]
Sep 09 21:15:33.863344 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.863325    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc3099-5cd8b49c98-6fwgs"]
Sep 09 21:15:33.873422 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.873400    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"]
Sep 09 21:15:33.878804 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:33.878784    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc3099-69445df59-md827"]
Sep 09 21:15:34.565855 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:34.565824    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6780cc41-21a3-4581-b2f5-8f252842a3f3" path="/var/lib/kubelet/pods/6780cc41-21a3-4581-b2f5-8f252842a3f3/volumes"
Sep 09 21:15:34.566217 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:34.566202    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73b3c213-f669-469c-aa17-aab28f9fc2c2" path="/var/lib/kubelet/pods/73b3c213-f669-469c-aa17-aab28f9fc2c2/volumes"
Sep 09 21:15:34.566521 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:34.566509    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da440b47-1ff1-4a5b-95c8-66bf6df40ab4" path="/var/lib/kubelet/pods/da440b47-1ff1-4a5b-95c8-66bf6df40ab4/volumes"
Sep 09 21:15:34.566826 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:34.566816    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e85a3441-f45b-47a6-acd0-aea38b2df7bf" path="/var/lib/kubelet/pods/e85a3441-f45b-47a6-acd0-aea38b2df7bf/volumes"
Sep 09 21:15:38.385776 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:38.385740    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:39.439982 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:39.439953    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:39.539501 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:39.539464    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:39.706787 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:39.706748    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:40.022859 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.022765    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"]
Sep 09 21:15:40.023351 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023332    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73b3c213-f669-469c-aa17-aab28f9fc2c2" containerName="payload-pre-processing"
Sep 09 21:15:40.023460 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023353    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="73b3c213-f669-469c-aa17-aab28f9fc2c2" containerName="payload-pre-processing"
Sep 09 21:15:40.023460 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023364    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85a3441-f45b-47a6-acd0-aea38b2df7bf" containerName="payload-processing"
Sep 09 21:15:40.023460 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023372    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85a3441-f45b-47a6-acd0-aea38b2df7bf" containerName="payload-processing"
Sep 09 21:15:40.023460 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023382    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da440b47-1ff1-4a5b-95c8-66bf6df40ab4" containerName="revoke-keys"
Sep 09 21:15:40.023460 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023391    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="da440b47-1ff1-4a5b-95c8-66bf6df40ab4" containerName="revoke-keys"
Sep 09 21:15:40.023711 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023512    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="73b3c213-f669-469c-aa17-aab28f9fc2c2" containerName="payload-pre-processing"
Sep 09 21:15:40.023711 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023528    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="da440b47-1ff1-4a5b-95c8-66bf6df40ab4" containerName="revoke-keys"
Sep 09 21:15:40.023711 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.023537    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e85a3441-f45b-47a6-acd0-aea38b2df7bf" containerName="payload-processing"
Sep 09 21:15:40.028643 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.028623    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"
Sep 09 21:15:40.031330 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.031299    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:15:40.033832 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.033805    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"]
Sep 09 21:15:40.096956 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.096917    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dd8877-852f-4be2-b877-90e2d13c0c30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j\" (UID: \"61dd8877-852f-4be2-b877-90e2d13c0c30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"
Sep 09 21:15:40.128909 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.128876    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:40.198345 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.198314    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dd8877-852f-4be2-b877-90e2d13c0c30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j\" (UID: \"61dd8877-852f-4be2-b877-90e2d13c0c30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"
Sep 09 21:15:40.198918 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.198898    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dd8877-852f-4be2-b877-90e2d13c0c30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j\" (UID: \"61dd8877-852f-4be2-b877-90e2d13c0c30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"
Sep 09 21:15:40.343354 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.343253    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"
Sep 09 21:15:40.616755 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.616658    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"]
Sep 09 21:15:40.618856 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:15:40.618821    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dd8877_852f_4be2_b877_90e2d13c0c30.slice/crio-34150239093fec6b7cdbb478a6a405ce1dbe85e9dbe0484bb2c47f83f84c887a WatchSource:0}: Error finding container 34150239093fec6b7cdbb478a6a405ce1dbe85e9dbe0484bb2c47f83f84c887a: Status 404 returned error can't find the container with id 34150239093fec6b7cdbb478a6a405ce1dbe85e9dbe0484bb2c47f83f84c887a
Sep 09 21:15:40.882776 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:40.882743    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j" event={"ID":"61dd8877-852f-4be2-b877-90e2d13c0c30","Type":"ContainerStarted","Data":"34150239093fec6b7cdbb478a6a405ce1dbe85e9dbe0484bb2c47f83f84c887a"}
Sep 09 21:15:41.891248 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:41.891206    2571 generic.go:356] "Generic (PLEG): container finished" podID="61dd8877-852f-4be2-b877-90e2d13c0c30" containerID="8ce3b870198c46f454eb4c651e927ab339ce4b4148a2778855b3be37c17f08d2" exitCode=0
Sep 09 21:15:41.891642 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:41.891290    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j" event={"ID":"61dd8877-852f-4be2-b877-90e2d13c0c30","Type":"ContainerDied","Data":"8ce3b870198c46f454eb4c651e927ab339ce4b4148a2778855b3be37c17f08d2"}
Sep 09 21:15:43.038045 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:43.038020    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"
Sep 09 21:15:43.126040 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:43.126007    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dd8877-852f-4be2-b877-90e2d13c0c30-maas-api-service-ca\") pod \"61dd8877-852f-4be2-b877-90e2d13c0c30\" (UID: \"61dd8877-852f-4be2-b877-90e2d13c0c30\") "
Sep 09 21:15:43.126424 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:43.126401    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61dd8877-852f-4be2-b877-90e2d13c0c30-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "61dd8877-852f-4be2-b877-90e2d13c0c30" (UID: "61dd8877-852f-4be2-b877-90e2d13c0c30"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:15:43.227388 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:43.227356    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/61dd8877-852f-4be2-b877-90e2d13c0c30-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:43.905444 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:43.905415    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"
Sep 09 21:15:43.905625 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:43.905455    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j" event={"ID":"61dd8877-852f-4be2-b877-90e2d13c0c30","Type":"ContainerDied","Data":"34150239093fec6b7cdbb478a6a405ce1dbe85e9dbe0484bb2c47f83f84c887a"}
Sep 09 21:15:43.905625 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:43.905502    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34150239093fec6b7cdbb478a6a405ce1dbe85e9dbe0484bb2c47f83f84c887a"
Sep 09 21:15:55.635601 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.635560    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:55.652145 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.652113    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"]
Sep 09 21:15:55.652374 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.652331    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t" podUID="71066f51-b807-46cb-b80a-a20a37ed3bcc" containerName="payload-processing" containerID="cri-o://05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac" gracePeriod=30
Sep 09 21:15:55.735825 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.735792    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:55.832469 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.832440    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"]
Sep 09 21:15:55.838278 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.838251    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bd9741-dbt9j"]
Sep 09 21:15:55.841004 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.840863    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:55.909043 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.908996    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:55.922591 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.922566    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:15:55.983217 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.983183    2571 generic.go:356] "Generic (PLEG): container finished" podID="71066f51-b807-46cb-b80a-a20a37ed3bcc" containerID="05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac" exitCode=0
Sep 09 21:15:55.983390 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.983230    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"
Sep 09 21:15:55.983390 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.983275    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t" event={"ID":"71066f51-b807-46cb-b80a-a20a37ed3bcc","Type":"ContainerDied","Data":"05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac"}
Sep 09 21:15:55.983390 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.983356    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t" event={"ID":"71066f51-b807-46cb-b80a-a20a37ed3bcc","Type":"ContainerDied","Data":"7a5a870bbfeebd09ac6fbfcc87e5c70533a84dd568abfb7d1f37d215e3444bce"}
Sep 09 21:15:55.983390 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.983385    2571 scope.go:117] "RemoveContainer" containerID="05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac"
Sep 09 21:15:55.992808 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.992790    2571 scope.go:117] "RemoveContainer" containerID="05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac"
Sep 09 21:15:55.993126 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:15:55.993071    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac\": container with ID starting with 05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac not found: ID does not exist" containerID="05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac"
Sep 09 21:15:55.993213 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:55.993140    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac"} err="failed to get container status \"05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac\": rpc error: code = NotFound desc = could not find container \"05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac\": container with ID starting with 05f2524d16b985fcc674fff89bdd6e9cdf858a0ba7d194518625c872b3adfaac not found: ID does not exist"
Sep 09 21:15:56.060476 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.060439    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2jcj\" (UniqueName: \"kubernetes.io/projected/71066f51-b807-46cb-b80a-a20a37ed3bcc-kube-api-access-z2jcj\") pod \"71066f51-b807-46cb-b80a-a20a37ed3bcc\" (UID: \"71066f51-b807-46cb-b80a-a20a37ed3bcc\") "
Sep 09 21:15:56.060655 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.060553    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71066f51-b807-46cb-b80a-a20a37ed3bcc-plugins-config-volume\") pod \"71066f51-b807-46cb-b80a-a20a37ed3bcc\" (UID: \"71066f51-b807-46cb-b80a-a20a37ed3bcc\") "
Sep 09 21:15:56.060929 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.060906    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71066f51-b807-46cb-b80a-a20a37ed3bcc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "71066f51-b807-46cb-b80a-a20a37ed3bcc" (UID: "71066f51-b807-46cb-b80a-a20a37ed3bcc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:15:56.062788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.062768    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71066f51-b807-46cb-b80a-a20a37ed3bcc-kube-api-access-z2jcj" (OuterVolumeSpecName: "kube-api-access-z2jcj") pod "71066f51-b807-46cb-b80a-a20a37ed3bcc" (UID: "71066f51-b807-46cb-b80a-a20a37ed3bcc"). InnerVolumeSpecName "kube-api-access-z2jcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:15:56.161636 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.161550    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/71066f51-b807-46cb-b80a-a20a37ed3bcc-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:56.161636 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.161579    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2jcj\" (UniqueName: \"kubernetes.io/projected/71066f51-b807-46cb-b80a-a20a37ed3bcc-kube-api-access-z2jcj\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:15:56.176943 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.176910    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:15:56.306611 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.306584    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"]
Sep 09 21:15:56.312762 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.312740    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bd9741-7ff5886947-gc92t"]
Sep 09 21:15:56.566134 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.566103    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61dd8877-852f-4be2-b877-90e2d13c0c30" path="/var/lib/kubelet/pods/61dd8877-852f-4be2-b877-90e2d13c0c30/volumes"
Sep 09 21:15:56.566463 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:56.566449    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71066f51-b807-46cb-b80a-a20a37ed3bcc" path="/var/lib/kubelet/pods/71066f51-b807-46cb-b80a-a20a37ed3bcc/volumes"
Sep 09 21:15:58.248847 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:15:58.248812    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:01.906784 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:01.906753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:02.007875 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.007839    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:02.124315 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.124278    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:02.453072 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.453039    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"]
Sep 09 21:16:02.453515 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.453502    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71066f51-b807-46cb-b80a-a20a37ed3bcc" containerName="payload-processing"
Sep 09 21:16:02.453561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.453517    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="71066f51-b807-46cb-b80a-a20a37ed3bcc" containerName="payload-processing"
Sep 09 21:16:02.453561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.453538    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61dd8877-852f-4be2-b877-90e2d13c0c30" containerName="revoke-keys"
Sep 09 21:16:02.453561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.453543    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="61dd8877-852f-4be2-b877-90e2d13c0c30" containerName="revoke-keys"
Sep 09 21:16:02.453656 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.453613    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="71066f51-b807-46cb-b80a-a20a37ed3bcc" containerName="payload-processing"
Sep 09 21:16:02.453656 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.453624    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="61dd8877-852f-4be2-b877-90e2d13c0c30" containerName="revoke-keys"
Sep 09 21:16:02.456831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.456816    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"
Sep 09 21:16:02.459818 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.459790    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:16:02.463336 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.463314    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"]
Sep 09 21:16:02.519920 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.519882    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54da6ad1-ee9a-41d1-a2b7-0947922b079c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj\" (UID: \"54da6ad1-ee9a-41d1-a2b7-0947922b079c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"
Sep 09 21:16:02.563647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.563593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:02.621395 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.621350    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54da6ad1-ee9a-41d1-a2b7-0947922b079c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj\" (UID: \"54da6ad1-ee9a-41d1-a2b7-0947922b079c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"
Sep 09 21:16:02.621965 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.621946    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54da6ad1-ee9a-41d1-a2b7-0947922b079c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj\" (UID: \"54da6ad1-ee9a-41d1-a2b7-0947922b079c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"
Sep 09 21:16:02.770564 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:02.770464    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"
Sep 09 21:16:03.044335 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:03.044297    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"]
Sep 09 21:16:03.045614 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:16:03.045583    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54da6ad1_ee9a_41d1_a2b7_0947922b079c.slice/crio-4c0f4e95da2d5587033629e5770a16d99971aaf9d969dd33177b0880ed2074f2 WatchSource:0}: Error finding container 4c0f4e95da2d5587033629e5770a16d99971aaf9d969dd33177b0880ed2074f2: Status 404 returned error can't find the container with id 4c0f4e95da2d5587033629e5770a16d99971aaf9d969dd33177b0880ed2074f2
Sep 09 21:16:04.047455 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:04.047420    2571 generic.go:356] "Generic (PLEG): container finished" podID="54da6ad1-ee9a-41d1-a2b7-0947922b079c" containerID="81ba05fac37995bc23343b08aa9d58211dc53e17aa2059bf8ae054fdff5b6a10" exitCode=0
Sep 09 21:16:04.047831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:04.047474    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj" event={"ID":"54da6ad1-ee9a-41d1-a2b7-0947922b079c","Type":"ContainerDied","Data":"81ba05fac37995bc23343b08aa9d58211dc53e17aa2059bf8ae054fdff5b6a10"}
Sep 09 21:16:04.047831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:04.047518    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj" event={"ID":"54da6ad1-ee9a-41d1-a2b7-0947922b079c","Type":"ContainerStarted","Data":"4c0f4e95da2d5587033629e5770a16d99971aaf9d969dd33177b0880ed2074f2"}
Sep 09 21:16:05.189943 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:05.189920    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"
Sep 09 21:16:05.247617 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:05.247583    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54da6ad1-ee9a-41d1-a2b7-0947922b079c-maas-api-service-ca\") pod \"54da6ad1-ee9a-41d1-a2b7-0947922b079c\" (UID: \"54da6ad1-ee9a-41d1-a2b7-0947922b079c\") "
Sep 09 21:16:05.247932 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:05.247910    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54da6ad1-ee9a-41d1-a2b7-0947922b079c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54da6ad1-ee9a-41d1-a2b7-0947922b079c" (UID: "54da6ad1-ee9a-41d1-a2b7-0947922b079c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:05.348738 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:05.348655    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54da6ad1-ee9a-41d1-a2b7-0947922b079c-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:16:05.873208 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:05.873173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:06.063752 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:06.063720    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj" event={"ID":"54da6ad1-ee9a-41d1-a2b7-0947922b079c","Type":"ContainerDied","Data":"4c0f4e95da2d5587033629e5770a16d99971aaf9d969dd33177b0880ed2074f2"}
Sep 09 21:16:06.063921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:06.063765    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c0f4e95da2d5587033629e5770a16d99971aaf9d969dd33177b0880ed2074f2"
Sep 09 21:16:06.063921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:06.063728    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"
Sep 09 21:16:12.576097 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:12.576037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:13.212738 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.212702    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:13.221867 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.221831    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"]
Sep 09 21:16:13.229022 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.228981    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-629537-29816475-gwgtb"]
Sep 09 21:16:13.246192 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.246162    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"]
Sep 09 21:16:13.246442 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.246414    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs" podUID="65c14ae5-74d2-42a3-89a1-ac7357970c12" containerName="payload-processing" containerID="cri-o://ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e" gracePeriod=30
Sep 09 21:16:13.313164 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.313124    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:13.408829 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.408797    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"]
Sep 09 21:16:13.412380 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.412351    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:13.414319 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.414300    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-629537-58pnj"]
Sep 09 21:16:13.471642 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.471554    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:13.502619 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.502595    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:16:13.571387 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.571352    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:13.628554 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.628513    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c14ae5-74d2-42a3-89a1-ac7357970c12-plugins-config-volume\") pod \"65c14ae5-74d2-42a3-89a1-ac7357970c12\" (UID: \"65c14ae5-74d2-42a3-89a1-ac7357970c12\") "
Sep 09 21:16:13.629029 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.628606    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxhj2\" (UniqueName: \"kubernetes.io/projected/65c14ae5-74d2-42a3-89a1-ac7357970c12-kube-api-access-cxhj2\") pod \"65c14ae5-74d2-42a3-89a1-ac7357970c12\" (UID: \"65c14ae5-74d2-42a3-89a1-ac7357970c12\") "
Sep 09 21:16:13.629029 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.628930    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c14ae5-74d2-42a3-89a1-ac7357970c12-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65c14ae5-74d2-42a3-89a1-ac7357970c12" (UID: "65c14ae5-74d2-42a3-89a1-ac7357970c12"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:13.629029 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.629014    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65c14ae5-74d2-42a3-89a1-ac7357970c12-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:16:13.631120 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.631095    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c14ae5-74d2-42a3-89a1-ac7357970c12-kube-api-access-cxhj2" (OuterVolumeSpecName: "kube-api-access-cxhj2") pod "65c14ae5-74d2-42a3-89a1-ac7357970c12" (UID: "65c14ae5-74d2-42a3-89a1-ac7357970c12"). InnerVolumeSpecName "kube-api-access-cxhj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:13.730149 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.730057    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxhj2\" (UniqueName: \"kubernetes.io/projected/65c14ae5-74d2-42a3-89a1-ac7357970c12-kube-api-access-cxhj2\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:16:13.764965 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:13.764922    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:14.116504 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.116413    2571 generic.go:356] "Generic (PLEG): container finished" podID="65c14ae5-74d2-42a3-89a1-ac7357970c12" containerID="ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e" exitCode=0
Sep 09 21:16:14.116504 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.116457    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"
Sep 09 21:16:14.116504 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.116495    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs" event={"ID":"65c14ae5-74d2-42a3-89a1-ac7357970c12","Type":"ContainerDied","Data":"ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e"}
Sep 09 21:16:14.116745 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.116544    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs" event={"ID":"65c14ae5-74d2-42a3-89a1-ac7357970c12","Type":"ContainerDied","Data":"94a2d68523f30cdd6816d7bd8f0e8d0f8135ff1ee6b47b34512d3608cee90fe3"}
Sep 09 21:16:14.116745 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.116566    2571 scope.go:117] "RemoveContainer" containerID="ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e"
Sep 09 21:16:14.127891 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.127873    2571 scope.go:117] "RemoveContainer" containerID="ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e"
Sep 09 21:16:14.128206 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:16:14.128188    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e\": container with ID starting with ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e not found: ID does not exist" containerID="ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e"
Sep 09 21:16:14.128258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.128215    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e"} err="failed to get container status \"ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e\": rpc error: code = NotFound desc = could not find container \"ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e\": container with ID starting with ae35ef412e542c009bf2281074530028fad09f7828af9fb5a90355017735889e not found: ID does not exist"
Sep 09 21:16:14.140224 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.140198    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"]
Sep 09 21:16:14.143877 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.143856    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-629537-744fbb9fd5-h22fs"]
Sep 09 21:16:14.567006 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.566973    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54da6ad1-ee9a-41d1-a2b7-0947922b079c" path="/var/lib/kubelet/pods/54da6ad1-ee9a-41d1-a2b7-0947922b079c/volumes"
Sep 09 21:16:14.567312 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.567299    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fa47000-0034-485d-8709-7e9813b42d81" path="/var/lib/kubelet/pods/5fa47000-0034-485d-8709-7e9813b42d81/volumes"
Sep 09 21:16:14.567584 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:14.567572    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c14ae5-74d2-42a3-89a1-ac7357970c12" path="/var/lib/kubelet/pods/65c14ae5-74d2-42a3-89a1-ac7357970c12/volumes"
Sep 09 21:16:15.700435 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:15.700385    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:16.717258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:16.717223    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:17.147199 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:17.147104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:17.515044 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:17.515010    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:17.993751 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:17.993708    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:19.814696 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:19.814667    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:19.914650 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:19.914610    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:20.076768 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.076685    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:20.401921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.401835    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"]
Sep 09 21:16:20.402323 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.402309    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c14ae5-74d2-42a3-89a1-ac7357970c12" containerName="payload-processing"
Sep 09 21:16:20.402373 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.402326    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c14ae5-74d2-42a3-89a1-ac7357970c12" containerName="payload-processing"
Sep 09 21:16:20.402373 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.402342    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54da6ad1-ee9a-41d1-a2b7-0947922b079c" containerName="revoke-keys"
Sep 09 21:16:20.402373 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.402348    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="54da6ad1-ee9a-41d1-a2b7-0947922b079c" containerName="revoke-keys"
Sep 09 21:16:20.402493 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.402441    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="54da6ad1-ee9a-41d1-a2b7-0947922b079c" containerName="revoke-keys"
Sep 09 21:16:20.402493 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.402450    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="65c14ae5-74d2-42a3-89a1-ac7357970c12" containerName="payload-processing"
Sep 09 21:16:20.405969 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.405953    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"
Sep 09 21:16:20.408423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.408406    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:16:20.421524 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.421493    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"]
Sep 09 21:16:20.494864 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.494831    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0708f11-ccdc-4144-8e4f-db0e4502166e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz\" (UID: \"a0708f11-ccdc-4144-8e4f-db0e4502166e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"
Sep 09 21:16:20.529179 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.529138    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:20.595860 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.595829    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0708f11-ccdc-4144-8e4f-db0e4502166e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz\" (UID: \"a0708f11-ccdc-4144-8e4f-db0e4502166e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"
Sep 09 21:16:20.596531 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.596510    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0708f11-ccdc-4144-8e4f-db0e4502166e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz\" (UID: \"a0708f11-ccdc-4144-8e4f-db0e4502166e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"
Sep 09 21:16:20.723179 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.723144    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"
Sep 09 21:16:20.741635 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.741601    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:20.830314 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.830282    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:20.999801 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:20.999767    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"]
Sep 09 21:16:21.000613 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:16:21.000586    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0708f11_ccdc_4144_8e4f_db0e4502166e.slice/crio-f7e980a924e7d43a7928d74bbe684a753dedfa9edbf27ca7251dc580d836e318 WatchSource:0}: Error finding container f7e980a924e7d43a7928d74bbe684a753dedfa9edbf27ca7251dc580d836e318: Status 404 returned error can't find the container with id f7e980a924e7d43a7928d74bbe684a753dedfa9edbf27ca7251dc580d836e318
Sep 09 21:16:21.166149 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:21.166120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz" event={"ID":"a0708f11-ccdc-4144-8e4f-db0e4502166e","Type":"ContainerStarted","Data":"f7e980a924e7d43a7928d74bbe684a753dedfa9edbf27ca7251dc580d836e318"}
Sep 09 21:16:21.989900 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:21.989836    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:22.090043 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.089925    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:22.174544 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.174512    2571 generic.go:356] "Generic (PLEG): container finished" podID="a0708f11-ccdc-4144-8e4f-db0e4502166e" containerID="cd634dfb70ffcb8b3d536013693a67a3ee50e3b6756b9528c333e93ad1d9f5fc" exitCode=0
Sep 09 21:16:22.174747 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.174565    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz" event={"ID":"a0708f11-ccdc-4144-8e4f-db0e4502166e","Type":"ContainerDied","Data":"cd634dfb70ffcb8b3d536013693a67a3ee50e3b6756b9528c333e93ad1d9f5fc"}
Sep 09 21:16:22.317732 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.317699    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:22.605477 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.605429    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:22.699506 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.699462    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:22.706680 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.706658    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"]
Sep 09 21:16:22.710355 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.710339    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:22.712885 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.712866    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-d7f624-dockercfg-xk9rk\""
Sep 09 21:16:22.712971 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.712923    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-d7f624\""
Sep 09 21:16:22.725446 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.725422    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"]
Sep 09 21:16:22.742288 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.742261    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"]
Sep 09 21:16:22.746277 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.746255    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:22.756384 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.756359    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"]
Sep 09 21:16:22.817937 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.817900    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gz9q\" (UniqueName: \"kubernetes.io/projected/221ebc00-7949-4e38-b480-a583cee36f24-kube-api-access-6gz9q\") pod \"payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh\" (UID: \"221ebc00-7949-4e38-b480-a583cee36f24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:22.818149 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.818027    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/221ebc00-7949-4e38-b480-a583cee36f24-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh\" (UID: \"221ebc00-7949-4e38-b480-a583cee36f24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:22.818149 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.818097    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c\" (UID: \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:22.818265 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.818163    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knmh2\" (UniqueName: \"kubernetes.io/projected/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-kube-api-access-knmh2\") pod \"payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c\" (UID: \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:22.919443 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.919342    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gz9q\" (UniqueName: \"kubernetes.io/projected/221ebc00-7949-4e38-b480-a583cee36f24-kube-api-access-6gz9q\") pod \"payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh\" (UID: \"221ebc00-7949-4e38-b480-a583cee36f24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:22.919443 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.919395    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/221ebc00-7949-4e38-b480-a583cee36f24-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh\" (UID: \"221ebc00-7949-4e38-b480-a583cee36f24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:22.919443 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.919438    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c\" (UID: \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:22.919744 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.919585    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knmh2\" (UniqueName: \"kubernetes.io/projected/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-kube-api-access-knmh2\") pod \"payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c\" (UID: \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:22.920262 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.920240    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c\" (UID: \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:22.920346 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.920264    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/221ebc00-7949-4e38-b480-a583cee36f24-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh\" (UID: \"221ebc00-7949-4e38-b480-a583cee36f24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:22.927570 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.927551    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gz9q\" (UniqueName: \"kubernetes.io/projected/221ebc00-7949-4e38-b480-a583cee36f24-kube-api-access-6gz9q\") pod \"payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh\" (UID: \"221ebc00-7949-4e38-b480-a583cee36f24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:22.927676 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:22.927657    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knmh2\" (UniqueName: \"kubernetes.io/projected/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-kube-api-access-knmh2\") pod \"payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c\" (UID: \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:23.020614 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:23.020573    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:23.058661 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:23.058625    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:23.303610 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:23.303584    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"]
Sep 09 21:16:23.303983 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:16:23.303962    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221ebc00_7949_4e38_b480_a583cee36f24.slice/crio-5f7caa890a4f70eaec7c250aec68592126cd252494cde2589a2e21db844e8805 WatchSource:0}: Error finding container 5f7caa890a4f70eaec7c250aec68592126cd252494cde2589a2e21db844e8805: Status 404 returned error can't find the container with id 5f7caa890a4f70eaec7c250aec68592126cd252494cde2589a2e21db844e8805
Sep 09 21:16:23.306841 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:23.306826    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"
Sep 09 21:16:23.355413 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:23.354573    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"]
Sep 09 21:16:23.360174 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:16:23.360147    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16cf2cb6_3931_4dd9_b2a1_de5b5e72b1bc.slice/crio-d2fbe63d7e1e70a0eec3163db6b18f5c2455272b8d538ab465b31ac611c640bf WatchSource:0}: Error finding container d2fbe63d7e1e70a0eec3163db6b18f5c2455272b8d538ab465b31ac611c640bf: Status 404 returned error can't find the container with id d2fbe63d7e1e70a0eec3163db6b18f5c2455272b8d538ab465b31ac611c640bf
Sep 09 21:16:23.424531 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:23.424502    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0708f11-ccdc-4144-8e4f-db0e4502166e-maas-api-service-ca\") pod \"a0708f11-ccdc-4144-8e4f-db0e4502166e\" (UID: \"a0708f11-ccdc-4144-8e4f-db0e4502166e\") "
Sep 09 21:16:23.424924 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:23.424898    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0708f11-ccdc-4144-8e4f-db0e4502166e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0708f11-ccdc-4144-8e4f-db0e4502166e" (UID: "a0708f11-ccdc-4144-8e4f-db0e4502166e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:23.525422 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:23.525384    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0708f11-ccdc-4144-8e4f-db0e4502166e-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:16:24.191243 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.191129    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c" event={"ID":"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc","Type":"ContainerStarted","Data":"0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff"}
Sep 09 21:16:24.191703 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.191257    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:24.191703 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.191286    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c" event={"ID":"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc","Type":"ContainerStarted","Data":"d2fbe63d7e1e70a0eec3163db6b18f5c2455272b8d538ab465b31ac611c640bf"}
Sep 09 21:16:24.192598 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.192583    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"
Sep 09 21:16:24.192669 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.192623    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz" event={"ID":"a0708f11-ccdc-4144-8e4f-db0e4502166e","Type":"ContainerDied","Data":"f7e980a924e7d43a7928d74bbe684a753dedfa9edbf27ca7251dc580d836e318"}
Sep 09 21:16:24.192729 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.192667    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7e980a924e7d43a7928d74bbe684a753dedfa9edbf27ca7251dc580d836e318"
Sep 09 21:16:24.194555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.194531    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh" event={"ID":"221ebc00-7949-4e38-b480-a583cee36f24","Type":"ContainerStarted","Data":"8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e"}
Sep 09 21:16:24.194736 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.194574    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh" event={"ID":"221ebc00-7949-4e38-b480-a583cee36f24","Type":"ContainerStarted","Data":"5f7caa890a4f70eaec7c250aec68592126cd252494cde2589a2e21db844e8805"}
Sep 09 21:16:24.194736 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.194691    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:24.209668 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.209627    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c" podStartSLOduration=2.209611705 podStartE2EDuration="2.209611705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:24.206292484 +0000 UTC m=+1678.176859200" watchObservedRunningTime="2026-09-09 21:16:24.209611705 +0000 UTC m=+1678.180178422"
Sep 09 21:16:24.224788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:24.224740    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh" podStartSLOduration=2.224725266 podStartE2EDuration="2.224725266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:24.222656725 +0000 UTC m=+1678.193223440" watchObservedRunningTime="2026-09-09 21:16:24.224725266 +0000 UTC m=+1678.195291980"
Sep 09 21:16:26.906719 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:26.906688    2571 scope.go:117] "RemoveContainer" containerID="1f0de9488ff621628206ff2e498918bb2e6497daab25a779423c2a51a083c69f"
Sep 09 21:16:30.652874 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:30.652837    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:31.242723 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.242688    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:31.245316 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.245292    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"]
Sep 09 21:16:31.250619 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.250595    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-459158-29816475-2cmcq"]
Sep 09 21:16:31.278520 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.278484    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"]
Sep 09 21:16:31.278718 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.278694    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx" podUID="3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe" containerName="payload-processing" containerID="cri-o://c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51" gracePeriod=30
Sep 09 21:16:31.340675 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.340627    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:31.439753 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.439705    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:31.451958 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.451919    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"]
Sep 09 21:16:31.454669 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.454647    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-459158-rwrpz"]
Sep 09 21:16:31.529620 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.529590    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:31.543766 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.543741    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:16:31.705879 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.705850    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhkqc\" (UniqueName: \"kubernetes.io/projected/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-kube-api-access-rhkqc\") pod \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\" (UID: \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\") "
Sep 09 21:16:31.706292 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.705902    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-plugins-config-volume\") pod \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\" (UID: \"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe\") "
Sep 09 21:16:31.706383 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.706356    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe" (UID: "3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:16:31.708146 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.708125    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-kube-api-access-rhkqc" (OuterVolumeSpecName: "kube-api-access-rhkqc") pod "3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe" (UID: "3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe"). InnerVolumeSpecName "kube-api-access-rhkqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:16:31.795858 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.795746    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:31.806921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.806896    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhkqc\" (UniqueName: \"kubernetes.io/projected/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-kube-api-access-rhkqc\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:16:31.806921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:31.806921    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:16:32.249334 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.249300    2571 generic.go:356] "Generic (PLEG): container finished" podID="3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe" containerID="c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51" exitCode=0
Sep 09 21:16:32.249570 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.249345    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"
Sep 09 21:16:32.249570 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.249354    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx" event={"ID":"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe","Type":"ContainerDied","Data":"c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51"}
Sep 09 21:16:32.249570 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.249395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx" event={"ID":"3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe","Type":"ContainerDied","Data":"01d4ee2c1feb3633850a461f864c21cc22df3e8d1a9e8c18f5e70761f7c9777e"}
Sep 09 21:16:32.249570 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.249416    2571 scope.go:117] "RemoveContainer" containerID="c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51"
Sep 09 21:16:32.269192 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.269160    2571 scope.go:117] "RemoveContainer" containerID="c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51"
Sep 09 21:16:32.269675 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:16:32.269653    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51\": container with ID starting with c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51 not found: ID does not exist" containerID="c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51"
Sep 09 21:16:32.269766 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.269680    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51"} err="failed to get container status \"c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51\": rpc error: code = NotFound desc = could not find container \"c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51\": container with ID starting with c668c809c5f75b3d491be44d589429d65eeb717090e62226904212b28b1c9d51 not found: ID does not exist"
Sep 09 21:16:32.277056 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.277028    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"]
Sep 09 21:16:32.280722 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.280695    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-459158-b4c7dc955-wgjxx"]
Sep 09 21:16:32.566526 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.566443    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe" path="/var/lib/kubelet/pods/3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe/volumes"
Sep 09 21:16:32.566753 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.566741    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45ae43fb-b3e1-4950-ab67-cee65ef4770d" path="/var/lib/kubelet/pods/45ae43fb-b3e1-4950-ab67-cee65ef4770d/volumes"
Sep 09 21:16:32.567007 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:32.566996    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0708f11-ccdc-4144-8e4f-db0e4502166e" path="/var/lib/kubelet/pods/a0708f11-ccdc-4144-8e4f-db0e4502166e/volumes"
Sep 09 21:16:35.204712 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:35.204685    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:16:35.205148 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:35.205003    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:16:37.912042 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:37.912004    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:38.011845 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:38.011806    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:38.196170 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:38.194108    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:43.019327 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:43.019294    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:45.642236 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:45.642200    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:46.313836 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:46.313783    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:46.715571 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:46.715540    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:46.815692 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:46.815657    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:50.121571 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:50.121487    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:50.209737 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:50.209679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:51.488982 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:51.488946    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:51.593853 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:51.593810    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:51.688747 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:51.688705    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:52.056321 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.056285    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:52.169488 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.169451    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:16:52.221674 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.221646    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"]
Sep 09 21:16:52.222166 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.222147    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe" containerName="payload-processing"
Sep 09 21:16:52.222215 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.222169    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe" containerName="payload-processing"
Sep 09 21:16:52.222215 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.222179    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0708f11-ccdc-4144-8e4f-db0e4502166e" containerName="revoke-keys"
Sep 09 21:16:52.222215 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.222184    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0708f11-ccdc-4144-8e4f-db0e4502166e" containerName="revoke-keys"
Sep 09 21:16:52.222312 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.222246    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0708f11-ccdc-4144-8e4f-db0e4502166e" containerName="revoke-keys"
Sep 09 21:16:52.222312 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.222264    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3e0aa2-d88d-466f-b48f-5e6b3ff1affe" containerName="payload-processing"
Sep 09 21:16:52.226850 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.226831    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:52.229646 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.229620    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-d7dd56\""
Sep 09 21:16:52.314741 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.314639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j\" (UID: \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:52.314894 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.314780    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptq97\" (UniqueName: \"kubernetes.io/projected/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-kube-api-access-ptq97\") pod \"payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j\" (UID: \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:52.323593 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.323559    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"]
Sep 09 21:16:52.415402 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.415366    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j\" (UID: \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:52.415604 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.415462    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptq97\" (UniqueName: \"kubernetes.io/projected/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-kube-api-access-ptq97\") pod \"payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j\" (UID: \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:52.416117 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.416063    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j\" (UID: \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:52.434423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.434397    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptq97\" (UniqueName: \"kubernetes.io/projected/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-kube-api-access-ptq97\") pod \"payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j\" (UID: \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:52.538100 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.538056    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:52.817463 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:52.817427    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"]
Sep 09 21:16:52.818671 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:16:52.818644    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b69aae_caeb_4d99_8cb2_6c2b77ca15e2.slice/crio-2bbadc10919090c1a2b6994ad636780684446f74575917e39486606e0325f414 WatchSource:0}: Error finding container 2bbadc10919090c1a2b6994ad636780684446f74575917e39486606e0325f414: Status 404 returned error can't find the container with id 2bbadc10919090c1a2b6994ad636780684446f74575917e39486606e0325f414
Sep 09 21:16:53.391286 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:53.391250    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j" event={"ID":"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2","Type":"ContainerStarted","Data":"e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173"}
Sep 09 21:16:53.391480 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:53.391302    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j" event={"ID":"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2","Type":"ContainerStarted","Data":"2bbadc10919090c1a2b6994ad636780684446f74575917e39486606e0325f414"}
Sep 09 21:16:53.391480 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:53.391336    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:16:53.407786 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:53.407726    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j" podStartSLOduration=1.407707744 podStartE2EDuration="1.407707744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:16:53.406492532 +0000 UTC m=+1707.377059246" watchObservedRunningTime="2026-09-09 21:16:53.407707744 +0000 UTC m=+1707.378274461"
Sep 09 21:16:56.486359 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:16:56.486324    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:17:04.399987 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:04.399955    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:17:06.602547 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:06.602509    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:17:13.757161 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:13.757101    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:17:15.056551 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:15.056511    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:17:18.026997 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:18.026965    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:17:18.145045 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:18.145015    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j_40b69aae-caeb-4d99-8cb2-6c2b77ca15e2/payload-processing/0.log"
Sep 09 21:17:34.915396 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:34.915355    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"]
Sep 09 21:17:34.919153 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:34.919130    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:34.922213 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:34.922186    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-d7f624\"/\"kube-root-ca.crt\""
Sep 09 21:17:34.922349 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:34.922186    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-d7f624\"/\"ipp-route-d7f624-kserve-self-signed-certs\""
Sep 09 21:17:34.922349 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:34.922189    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-d7f624\"/\"default-dockercfg-8bzbg\""
Sep 09 21:17:34.922349 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:34.922189    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-d7f624\"/\"openshift-service-ca.crt\""
Sep 09 21:17:34.930290 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:34.930264    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"]
Sep 09 21:17:34.943522 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:34.943492    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:17:35.035306 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.035271    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tls-certs\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.035511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.035317    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgzq5\" (UniqueName: \"kubernetes.io/projected/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-kube-api-access-vgzq5\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.035511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.035398    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-home\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.035511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.035439    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tmp-dir\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.035511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.035459    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-model-cache\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.035753 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.035548    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-dshm\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136297 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136248    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tls-certs\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136508 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136315    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgzq5\" (UniqueName: \"kubernetes.io/projected/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-kube-api-access-vgzq5\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136508 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-home\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136508 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136413    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tmp-dir\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136508 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136436    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-model-cache\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136732 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136515    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-dshm\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136810 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136786    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-home\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136871 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136854    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tmp-dir\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.136928 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.136867    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-model-cache\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.138947 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.138922    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tls-certs\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.139135 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.139115    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-dshm\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.144597 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.144567    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgzq5\" (UniqueName: \"kubernetes.io/projected/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-kube-api-access-vgzq5\") pod \"ipp-route-d7f624-kserve-589d47744c-m8xg8\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") " pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.231638 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.231598    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:35.505561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.505528    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"]
Sep 09 21:17:35.507453 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:17:35.507430    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode275d100_7e42_4f3b_a7e8_1c8c7d8e0cdf.slice/crio-0a49430d44f23118bdd913892c4ede3049f92b1289194b806393e8e79838e4d9 WatchSource:0}: Error finding container 0a49430d44f23118bdd913892c4ede3049f92b1289194b806393e8e79838e4d9: Status 404 returned error can't find the container with id 0a49430d44f23118bdd913892c4ede3049f92b1289194b806393e8e79838e4d9
Sep 09 21:17:35.667818 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.667779    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8" event={"ID":"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf","Type":"ContainerStarted","Data":"abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603"}
Sep 09 21:17:35.667818 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.667823    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8" event={"ID":"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf","Type":"ContainerStarted","Data":"0a49430d44f23118bdd913892c4ede3049f92b1289194b806393e8e79838e4d9"}
Sep 09 21:17:35.685028 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:35.684971    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8" podStartSLOduration=1.684952874 podStartE2EDuration="1.684952874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:17:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:17:35.683069379 +0000 UTC m=+1749.653636096" watchObservedRunningTime="2026-09-09 21:17:35.684952874 +0000 UTC m=+1749.655519590"
Sep 09 21:17:45.232000 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:45.231954    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:45.232429 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:45.232051    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:45.245014 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:45.244983    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:45.741764 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:45.741732    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:17:56.866262 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:56.866227    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c_16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc/payload-processing/0.log"
Sep 09 21:17:56.983281 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:17:56.983226    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:00.113255 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:00.113222    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:03.240472 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:03.240441    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:06.371358 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:06.371328    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:09.493004 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:09.492962    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:12.618410 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:12.618364    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:15.740591 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:15.740557    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:18.865811 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:18.865736    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:21.989842 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:21.989804    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:25.121674 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:25.121646    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:18:26.968541 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:26.968507    2571 scope.go:117] "RemoveContainer" containerID="edb9320f1ff1029a946e18f264b662352a08936af6b8fa6ce57a8c695a33825f"
Sep 09 21:18:28.248301 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:28.248260    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j_40b69aae-caeb-4d99-8cb2-6c2b77ca15e2/payload-processing/0.log"
Sep 09 21:18:28.515679 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:28.515582    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:28.887007 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:28.886911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:29.037331 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:29.037293    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:31.646071 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:31.646034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:31.747310 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:31.747270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:32.851515 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:32.851475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:32.951312 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:32.951254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:33.172865 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.172747    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:33.368216 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.368186    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:33.669647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.669614    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"]
Sep 09 21:18:33.673544 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.673524    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:33.675863 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.675838    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-8d0f62\""
Sep 09 21:18:33.676190 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.676172    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-8d0f62-dockercfg-zrwzc\""
Sep 09 21:18:33.684948 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.684924    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"]
Sep 09 21:18:33.820404 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.820370    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6246a249-6b89-4c62-995d-7d87ae61571b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2\" (UID: \"6246a249-6b89-4c62-995d-7d87ae61571b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:33.820404 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.820418    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctt7p\" (UniqueName: \"kubernetes.io/projected/6246a249-6b89-4c62-995d-7d87ae61571b-kube-api-access-ctt7p\") pod \"payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2\" (UID: \"6246a249-6b89-4c62-995d-7d87ae61571b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:33.921782 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.921695    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6246a249-6b89-4c62-995d-7d87ae61571b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2\" (UID: \"6246a249-6b89-4c62-995d-7d87ae61571b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:33.921782 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.921740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctt7p\" (UniqueName: \"kubernetes.io/projected/6246a249-6b89-4c62-995d-7d87ae61571b-kube-api-access-ctt7p\") pod \"payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2\" (UID: \"6246a249-6b89-4c62-995d-7d87ae61571b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:33.922511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.922489    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6246a249-6b89-4c62-995d-7d87ae61571b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2\" (UID: \"6246a249-6b89-4c62-995d-7d87ae61571b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:33.930404 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.930368    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctt7p\" (UniqueName: \"kubernetes.io/projected/6246a249-6b89-4c62-995d-7d87ae61571b-kube-api-access-ctt7p\") pod \"payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2\" (UID: \"6246a249-6b89-4c62-995d-7d87ae61571b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:33.985061 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:33.985023    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:34.270141 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:34.270115    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"]
Sep 09 21:18:34.272188 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:18:34.272153    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6246a249_6b89_4c62_995d_7d87ae61571b.slice/crio-8387267776c3c91a7fc9bf6fbadb2c069a6dc7622fe16ec8beee1574d4badef7 WatchSource:0}: Error finding container 8387267776c3c91a7fc9bf6fbadb2c069a6dc7622fe16ec8beee1574d4badef7: Status 404 returned error can't find the container with id 8387267776c3c91a7fc9bf6fbadb2c069a6dc7622fe16ec8beee1574d4badef7
Sep 09 21:18:35.057966 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:35.057918    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2" event={"ID":"6246a249-6b89-4c62-995d-7d87ae61571b","Type":"ContainerStarted","Data":"002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d"}
Sep 09 21:18:35.058465 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:35.057994    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2" event={"ID":"6246a249-6b89-4c62-995d-7d87ae61571b","Type":"ContainerStarted","Data":"8387267776c3c91a7fc9bf6fbadb2c069a6dc7622fe16ec8beee1574d4badef7"}
Sep 09 21:18:35.058465 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:35.058118    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:35.111618 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:35.111496    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2" podStartSLOduration=2.111478429 podStartE2EDuration="2.111478429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:18:35.109543589 +0000 UTC m=+1809.080110307" watchObservedRunningTime="2026-09-09 21:18:35.111478429 +0000 UTC m=+1809.082045145"
Sep 09 21:18:35.483682 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:35.483644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:46.066867 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:46.066832    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:18:56.700335 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:56.700301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:18:57.396678 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.396641    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"]
Sep 09 21:18:57.400571 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.400552    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"
Sep 09 21:18:57.403268 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.403236    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:18:57.409151 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.409119    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"]
Sep 09 21:18:57.449433 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.449393    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8c3975-54e7-4599-975e-638f329a7a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9\" (UID: \"1c8c3975-54e7-4599-975e-638f329a7a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"
Sep 09 21:18:57.550401 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.550356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8c3975-54e7-4599-975e-638f329a7a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9\" (UID: \"1c8c3975-54e7-4599-975e-638f329a7a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"
Sep 09 21:18:57.551001 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.550981    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8c3975-54e7-4599-975e-638f329a7a60-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9\" (UID: \"1c8c3975-54e7-4599-975e-638f329a7a60\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"
Sep 09 21:18:57.713500 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.713466    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"
Sep 09 21:18:57.987998 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:57.987974    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"]
Sep 09 21:18:57.990013 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:18:57.989990    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8c3975_54e7_4599_975e_638f329a7a60.slice/crio-c7512b50e8589a4e60c1de32538512dd49fb1bb771b77196a7b379b664df1813 WatchSource:0}: Error finding container c7512b50e8589a4e60c1de32538512dd49fb1bb771b77196a7b379b664df1813: Status 404 returned error can't find the container with id c7512b50e8589a4e60c1de32538512dd49fb1bb771b77196a7b379b664df1813
Sep 09 21:18:58.209392 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:58.209359    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9" event={"ID":"1c8c3975-54e7-4599-975e-638f329a7a60","Type":"ContainerStarted","Data":"c7512b50e8589a4e60c1de32538512dd49fb1bb771b77196a7b379b664df1813"}
Sep 09 21:18:59.218062 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:59.218022    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c8c3975-54e7-4599-975e-638f329a7a60" containerID="a891fcab2b757170cafa8110f0603372cd22b133e04d2bb9b6b9663c55cd75cb" exitCode=0
Sep 09 21:18:59.218497 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:18:59.218110    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9" event={"ID":"1c8c3975-54e7-4599-975e-638f329a7a60","Type":"ContainerDied","Data":"a891fcab2b757170cafa8110f0603372cd22b133e04d2bb9b6b9663c55cd75cb"}
Sep 09 21:19:00.374116 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:00.374072    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"
Sep 09 21:19:00.482990 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:00.482899    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8c3975-54e7-4599-975e-638f329a7a60-maas-api-service-ca\") pod \"1c8c3975-54e7-4599-975e-638f329a7a60\" (UID: \"1c8c3975-54e7-4599-975e-638f329a7a60\") "
Sep 09 21:19:00.483339 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:00.483319    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c8c3975-54e7-4599-975e-638f329a7a60-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1c8c3975-54e7-4599-975e-638f329a7a60" (UID: "1c8c3975-54e7-4599-975e-638f329a7a60"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:19:00.584603 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:00.584569    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c8c3975-54e7-4599-975e-638f329a7a60-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:01.238039 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:01.238009    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"
Sep 09 21:19:01.238248 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:01.238011    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9" event={"ID":"1c8c3975-54e7-4599-975e-638f329a7a60","Type":"ContainerDied","Data":"c7512b50e8589a4e60c1de32538512dd49fb1bb771b77196a7b379b664df1813"}
Sep 09 21:19:01.238248 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:01.238146    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7512b50e8589a4e60c1de32538512dd49fb1bb771b77196a7b379b664df1813"
Sep 09 21:19:03.382850 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.382816    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:03.421108 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.421045    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"]
Sep 09 21:19:03.421358 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.421326    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2" podUID="6246a249-6b89-4c62-995d-7d87ae61571b" containerName="payload-processing" containerID="cri-o://002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d" gracePeriod=30
Sep 09 21:19:03.485134 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.485097    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:03.583872 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.583836    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:03.607669 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.607634    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"]
Sep 09 21:19:03.613992 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.613012    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-8d0f62-gt9s9"]
Sep 09 21:19:03.681620 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.681589    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:03.687693 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.687669    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:19:03.818135 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.818063    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctt7p\" (UniqueName: \"kubernetes.io/projected/6246a249-6b89-4c62-995d-7d87ae61571b-kube-api-access-ctt7p\") pod \"6246a249-6b89-4c62-995d-7d87ae61571b\" (UID: \"6246a249-6b89-4c62-995d-7d87ae61571b\") "
Sep 09 21:19:03.818323 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.818222    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6246a249-6b89-4c62-995d-7d87ae61571b-plugins-config-volume\") pod \"6246a249-6b89-4c62-995d-7d87ae61571b\" (UID: \"6246a249-6b89-4c62-995d-7d87ae61571b\") "
Sep 09 21:19:03.818600 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.818570    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6246a249-6b89-4c62-995d-7d87ae61571b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6246a249-6b89-4c62-995d-7d87ae61571b" (UID: "6246a249-6b89-4c62-995d-7d87ae61571b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:19:03.820469 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.820447    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6246a249-6b89-4c62-995d-7d87ae61571b-kube-api-access-ctt7p" (OuterVolumeSpecName: "kube-api-access-ctt7p") pod "6246a249-6b89-4c62-995d-7d87ae61571b" (UID: "6246a249-6b89-4c62-995d-7d87ae61571b"). InnerVolumeSpecName "kube-api-access-ctt7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:19:03.919973 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.919879    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctt7p\" (UniqueName: \"kubernetes.io/projected/6246a249-6b89-4c62-995d-7d87ae61571b-kube-api-access-ctt7p\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:03.919973 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.919911    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6246a249-6b89-4c62-995d-7d87ae61571b-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:03.976915 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:03.976881    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:04.258648 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.258617    2571 generic.go:356] "Generic (PLEG): container finished" podID="6246a249-6b89-4c62-995d-7d87ae61571b" containerID="002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d" exitCode=0
Sep 09 21:19:04.258839 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.258659    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"
Sep 09 21:19:04.258839 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.258703    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2" event={"ID":"6246a249-6b89-4c62-995d-7d87ae61571b","Type":"ContainerDied","Data":"002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d"}
Sep 09 21:19:04.258839 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.258753    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2" event={"ID":"6246a249-6b89-4c62-995d-7d87ae61571b","Type":"ContainerDied","Data":"8387267776c3c91a7fc9bf6fbadb2c069a6dc7622fe16ec8beee1574d4badef7"}
Sep 09 21:19:04.258839 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.258783    2571 scope.go:117] "RemoveContainer" containerID="002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d"
Sep 09 21:19:04.269451 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.269433    2571 scope.go:117] "RemoveContainer" containerID="002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d"
Sep 09 21:19:04.269755 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:19:04.269739    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d\": container with ID starting with 002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d not found: ID does not exist" containerID="002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d"
Sep 09 21:19:04.269819 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.269767    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d"} err="failed to get container status \"002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d\": rpc error: code = NotFound desc = could not find container \"002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d\": container with ID starting with 002c103a7f8b12070e603e5b18e55fba53951b1544527109ccbad8da8c297e1d not found: ID does not exist"
Sep 09 21:19:04.282682 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.282640    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"]
Sep 09 21:19:04.290847 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.290816    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8d0f62-56cbbd6b79-4ztn2"]
Sep 09 21:19:04.567126 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.567027    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c8c3975-54e7-4599-975e-638f329a7a60" path="/var/lib/kubelet/pods/1c8c3975-54e7-4599-975e-638f329a7a60/volumes"
Sep 09 21:19:04.567494 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:04.567463    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6246a249-6b89-4c62-995d-7d87ae61571b" path="/var/lib/kubelet/pods/6246a249-6b89-4c62-995d-7d87ae61571b/volumes"
Sep 09 21:19:10.162459 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:10.162422    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:10.263576 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:10.263541    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:10.408156 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:10.408118    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:10.570031 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:10.569992    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:13.961285 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.961250    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"]
Sep 09 21:19:13.961912 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.961891    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6246a249-6b89-4c62-995d-7d87ae61571b" containerName="payload-processing"
Sep 09 21:19:13.962006 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.961915    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6246a249-6b89-4c62-995d-7d87ae61571b" containerName="payload-processing"
Sep 09 21:19:13.962006 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.961965    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c8c3975-54e7-4599-975e-638f329a7a60" containerName="revoke-keys"
Sep 09 21:19:13.962006 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.961974    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c8c3975-54e7-4599-975e-638f329a7a60" containerName="revoke-keys"
Sep 09 21:19:13.962192 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.962106    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6246a249-6b89-4c62-995d-7d87ae61571b" containerName="payload-processing"
Sep 09 21:19:13.962192 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.962125    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c8c3975-54e7-4599-975e-638f329a7a60" containerName="revoke-keys"
Sep 09 21:19:13.966696 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.966674    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"
Sep 09 21:19:13.969234 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.969209    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:19:13.974419 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:13.974386    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"]
Sep 09 21:19:14.082536 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:14.082506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:14.123775 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:14.123740    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ef5458-58b0-412f-ba21-0d30e1f05310-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w\" (UID: \"c8ef5458-58b0-412f-ba21-0d30e1f05310\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"
Sep 09 21:19:14.224980 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:14.224891    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ef5458-58b0-412f-ba21-0d30e1f05310-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w\" (UID: \"c8ef5458-58b0-412f-ba21-0d30e1f05310\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"
Sep 09 21:19:14.225563 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:14.225544    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ef5458-58b0-412f-ba21-0d30e1f05310-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w\" (UID: \"c8ef5458-58b0-412f-ba21-0d30e1f05310\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"
Sep 09 21:19:14.280885 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:14.280847    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"
Sep 09 21:19:14.567441 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:19:14.567415    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ef5458_58b0_412f_ba21_0d30e1f05310.slice/crio-9f8bb0b1209a40cdc050a14fc60e087b392a61c99ed567d1147b1c8b8fc00ffc WatchSource:0}: Error finding container 9f8bb0b1209a40cdc050a14fc60e087b392a61c99ed567d1147b1c8b8fc00ffc: Status 404 returned error can't find the container with id 9f8bb0b1209a40cdc050a14fc60e087b392a61c99ed567d1147b1c8b8fc00ffc
Sep 09 21:19:14.568513 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:14.568489    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"]
Sep 09 21:19:15.340749 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:15.340707    2571 generic.go:356] "Generic (PLEG): container finished" podID="c8ef5458-58b0-412f-ba21-0d30e1f05310" containerID="740217bad8aa8394160c782dc44030c1eab6531a5cbd0deb1e68d38ab54c8b09" exitCode=0
Sep 09 21:19:15.341214 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:15.340775    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w" event={"ID":"c8ef5458-58b0-412f-ba21-0d30e1f05310","Type":"ContainerDied","Data":"740217bad8aa8394160c782dc44030c1eab6531a5cbd0deb1e68d38ab54c8b09"}
Sep 09 21:19:15.341214 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:15.340868    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w" event={"ID":"c8ef5458-58b0-412f-ba21-0d30e1f05310","Type":"ContainerStarted","Data":"9f8bb0b1209a40cdc050a14fc60e087b392a61c99ed567d1147b1c8b8fc00ffc"}
Sep 09 21:19:15.618831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:15.618735    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:16.494903 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:16.494874    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"
Sep 09 21:19:16.543739 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:16.543703    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ef5458-58b0-412f-ba21-0d30e1f05310-maas-api-service-ca\") pod \"c8ef5458-58b0-412f-ba21-0d30e1f05310\" (UID: \"c8ef5458-58b0-412f-ba21-0d30e1f05310\") "
Sep 09 21:19:16.544053 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:16.544028    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ef5458-58b0-412f-ba21-0d30e1f05310-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8ef5458-58b0-412f-ba21-0d30e1f05310" (UID: "c8ef5458-58b0-412f-ba21-0d30e1f05310"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:19:16.645204 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:16.645098    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8ef5458-58b0-412f-ba21-0d30e1f05310-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:17.355272 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:17.355239    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"
Sep 09 21:19:17.355272 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:17.355260    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w" event={"ID":"c8ef5458-58b0-412f-ba21-0d30e1f05310","Type":"ContainerDied","Data":"9f8bb0b1209a40cdc050a14fc60e087b392a61c99ed567d1147b1c8b8fc00ffc"}
Sep 09 21:19:17.355490 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:17.355345    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f8bb0b1209a40cdc050a14fc60e087b392a61c99ed567d1147b1c8b8fc00ffc"
Sep 09 21:19:29.587179 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.587134    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:29.618201 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.618006    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"]
Sep 09 21:19:29.618400 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.618279    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c" podUID="16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc" containerName="payload-processing" containerID="cri-o://0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff" gracePeriod=30
Sep 09 21:19:29.622478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.622430    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"]
Sep 09 21:19:29.622729 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.622684    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh" podUID="221ebc00-7949-4e38-b480-a583cee36f24" containerName="payload-pre-processing" containerID="cri-o://8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e" gracePeriod=30
Sep 09 21:19:29.669197 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.669159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:29.764775 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.764721    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:29.823658 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.823612    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"]
Sep 09 21:19:29.826262 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.826241    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-d7f624-qvx7w"]
Sep 09 21:19:29.865596 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.865498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:29.899636 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.899609    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:19:29.902735 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.902717    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:19:29.971836 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.971803    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knmh2\" (UniqueName: \"kubernetes.io/projected/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-kube-api-access-knmh2\") pod \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\" (UID: \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\") "
Sep 09 21:19:29.971836 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.971842    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/221ebc00-7949-4e38-b480-a583cee36f24-plugins-config-volume\") pod \"221ebc00-7949-4e38-b480-a583cee36f24\" (UID: \"221ebc00-7949-4e38-b480-a583cee36f24\") "
Sep 09 21:19:29.972117 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.971865    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gz9q\" (UniqueName: \"kubernetes.io/projected/221ebc00-7949-4e38-b480-a583cee36f24-kube-api-access-6gz9q\") pod \"221ebc00-7949-4e38-b480-a583cee36f24\" (UID: \"221ebc00-7949-4e38-b480-a583cee36f24\") "
Sep 09 21:19:29.972117 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.971901    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-plugins-config-volume\") pod \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\" (UID: \"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc\") "
Sep 09 21:19:29.972289 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.972257    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221ebc00-7949-4e38-b480-a583cee36f24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "221ebc00-7949-4e38-b480-a583cee36f24" (UID: "221ebc00-7949-4e38-b480-a583cee36f24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:19:29.972422 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.972304    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc" (UID: "16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:19:29.974228 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.974204    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/221ebc00-7949-4e38-b480-a583cee36f24-kube-api-access-6gz9q" (OuterVolumeSpecName: "kube-api-access-6gz9q") pod "221ebc00-7949-4e38-b480-a583cee36f24" (UID: "221ebc00-7949-4e38-b480-a583cee36f24"). InnerVolumeSpecName "kube-api-access-6gz9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:19:29.974310 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:29.974234    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-kube-api-access-knmh2" (OuterVolumeSpecName: "kube-api-access-knmh2") pod "16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc" (UID: "16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc"). InnerVolumeSpecName "kube-api-access-knmh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:19:30.073450 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.073402    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knmh2\" (UniqueName: \"kubernetes.io/projected/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-kube-api-access-knmh2\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:30.073450 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.073448    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/221ebc00-7949-4e38-b480-a583cee36f24-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:30.073450 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.073460    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gz9q\" (UniqueName: \"kubernetes.io/projected/221ebc00-7949-4e38-b480-a583cee36f24-kube-api-access-6gz9q\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:30.073450 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.073468    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:30.218088 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.218051    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:30.442492 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.442451    2571 generic.go:356] "Generic (PLEG): container finished" podID="221ebc00-7949-4e38-b480-a583cee36f24" containerID="8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e" exitCode=0
Sep 09 21:19:30.442677 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.442502    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"
Sep 09 21:19:30.442677 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.442541    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh" event={"ID":"221ebc00-7949-4e38-b480-a583cee36f24","Type":"ContainerDied","Data":"8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e"}
Sep 09 21:19:30.442677 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.442597    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh" event={"ID":"221ebc00-7949-4e38-b480-a583cee36f24","Type":"ContainerDied","Data":"5f7caa890a4f70eaec7c250aec68592126cd252494cde2589a2e21db844e8805"}
Sep 09 21:19:30.442677 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.442619    2571 scope.go:117] "RemoveContainer" containerID="8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e"
Sep 09 21:19:30.444256 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.444220    2571 generic.go:356] "Generic (PLEG): container finished" podID="16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc" containerID="0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff" exitCode=0
Sep 09 21:19:30.444392 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.444261    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"
Sep 09 21:19:30.444392 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.444264    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c" event={"ID":"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc","Type":"ContainerDied","Data":"0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff"}
Sep 09 21:19:30.444511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.444442    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c" event={"ID":"16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc","Type":"ContainerDied","Data":"d2fbe63d7e1e70a0eec3163db6b18f5c2455272b8d538ab465b31ac611c640bf"}
Sep 09 21:19:30.455013 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.454988    2571 scope.go:117] "RemoveContainer" containerID="8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e"
Sep 09 21:19:30.455345 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:19:30.455327    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e\": container with ID starting with 8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e not found: ID does not exist" containerID="8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e"
Sep 09 21:19:30.455397 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.455355    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e"} err="failed to get container status \"8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e\": rpc error: code = NotFound desc = could not find container \"8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e\": container with ID starting with 8cb2425b3a5f588d2761b3b558c944b170de22dbbd075c134292fd12fbafd86e not found: ID does not exist"
Sep 09 21:19:30.455397 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.455371    2571 scope.go:117] "RemoveContainer" containerID="0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff"
Sep 09 21:19:30.464627 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.464607    2571 scope.go:117] "RemoveContainer" containerID="0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff"
Sep 09 21:19:30.464923 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:19:30.464898    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff\": container with ID starting with 0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff not found: ID does not exist" containerID="0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff"
Sep 09 21:19:30.464991 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.464937    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff"} err="failed to get container status \"0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff\": rpc error: code = NotFound desc = could not find container \"0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff\": container with ID starting with 0b1233b41a7da13ca63efa365c56f22ea81e22d636d374ef5cba8d2e8ee138ff not found: ID does not exist"
Sep 09 21:19:30.469420 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.469352    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"]
Sep 09 21:19:30.476026 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.475998    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7f624-5b4c77cf4-fwbxh"]
Sep 09 21:19:30.487806 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.487780    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"]
Sep 09 21:19:30.492538 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.492505    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7f624-7d7fbdf899-cm98c"]
Sep 09 21:19:30.566980 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.566941    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc" path="/var/lib/kubelet/pods/16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc/volumes"
Sep 09 21:19:30.567378 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.567364    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="221ebc00-7949-4e38-b480-a583cee36f24" path="/var/lib/kubelet/pods/221ebc00-7949-4e38-b480-a583cee36f24/volumes"
Sep 09 21:19:30.567661 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:30.567649    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ef5458-58b0-412f-ba21-0d30e1f05310" path="/var/lib/kubelet/pods/c8ef5458-58b0-412f-ba21-0d30e1f05310/volumes"
Sep 09 21:19:35.207201 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:35.207161    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"]
Sep 09 21:19:35.257765 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:35.257726    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:19:50.228264 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.228168    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8" podUID="e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" containerName="main" containerID="cri-o://abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603" gracePeriod=45
Sep 09 21:19:50.481126 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.481062    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:19:50.582957 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.582928    2571 generic.go:356] "Generic (PLEG): container finished" podID="e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" containerID="abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603" exitCode=0
Sep 09 21:19:50.583154 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.582968    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"
Sep 09 21:19:50.583154 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.583010    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8" event={"ID":"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf","Type":"ContainerDied","Data":"abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603"}
Sep 09 21:19:50.583154 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.583051    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8" event={"ID":"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf","Type":"ContainerDied","Data":"0a49430d44f23118bdd913892c4ede3049f92b1289194b806393e8e79838e4d9"}
Sep 09 21:19:50.583154 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.583068    2571 scope.go:117] "RemoveContainer" containerID="abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603"
Sep 09 21:19:50.592453 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.592437    2571 scope.go:117] "RemoveContainer" containerID="abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603"
Sep 09 21:19:50.592706 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:19:50.592691    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603\": container with ID starting with abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603 not found: ID does not exist" containerID="abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603"
Sep 09 21:19:50.592750 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.592714    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603"} err="failed to get container status \"abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603\": rpc error: code = NotFound desc = could not find container \"abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603\": container with ID starting with abdf2682659375022c687e35ca3fbaae0ddcb4f34ed75db18a8eee8922b39603 not found: ID does not exist"
Sep 09 21:19:50.679258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679225    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tls-certs\") pod \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") "
Sep 09 21:19:50.679258 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679257    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-dshm\") pod \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") "
Sep 09 21:19:50.679514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679320    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tmp-dir\") pod \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") "
Sep 09 21:19:50.679514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679395    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgzq5\" (UniqueName: \"kubernetes.io/projected/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-kube-api-access-vgzq5\") pod \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") "
Sep 09 21:19:50.679514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679422    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-home\") pod \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") "
Sep 09 21:19:50.679514 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679456    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-model-cache\") pod \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\" (UID: \"e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf\") "
Sep 09 21:19:50.679714 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679684    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" (UID: "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:19:50.679714 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679678    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-home" (OuterVolumeSpecName: "home") pod "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" (UID: "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:19:50.679835 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679742    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-model-cache" (OuterVolumeSpecName: "model-cache") pod "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" (UID: "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:19:50.679888 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679858    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-home\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:50.679888 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679875    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-model-cache\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:50.679994 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.679889    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tmp-dir\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:50.681751 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.681721    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-dshm" (OuterVolumeSpecName: "dshm") pod "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" (UID: "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:19:50.681751 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.681744    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-kube-api-access-vgzq5" (OuterVolumeSpecName: "kube-api-access-vgzq5") pod "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" (UID: "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf"). InnerVolumeSpecName "kube-api-access-vgzq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:19:50.681939 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.681788    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" (UID: "e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:19:50.781151 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.781026    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-tls-certs\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:50.781151 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.781055    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-dshm\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:50.781151 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.781065    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgzq5\" (UniqueName: \"kubernetes.io/projected/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf-kube-api-access-vgzq5\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:19:50.906032 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.905996    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"]
Sep 09 21:19:50.909933 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:50.909906    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-d7f624/ipp-route-d7f624-kserve-589d47744c-m8xg8"]
Sep 09 21:19:52.573912 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:19:52.573865    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" path="/var/lib/kubelet/pods/e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf/volumes"
Sep 09 21:20:02.794478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:02.794438    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:12.034251 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:12.034209    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:13.250163 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:13.250124    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:19.149530 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.149492    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:19.249445 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.249413    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:19.442107 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.441986    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:19.780115 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780067    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"]
Sep 09 21:20:19.780557 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780544    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc" containerName="payload-processing"
Sep 09 21:20:19.780603 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780559    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc" containerName="payload-processing"
Sep 09 21:20:19.780603 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780572    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ef5458-58b0-412f-ba21-0d30e1f05310" containerName="revoke-keys"
Sep 09 21:20:19.780603 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780578    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ef5458-58b0-412f-ba21-0d30e1f05310" containerName="revoke-keys"
Sep 09 21:20:19.780603 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780594    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" containerName="main"
Sep 09 21:20:19.780603 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780599    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" containerName="main"
Sep 09 21:20:19.780761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780611    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="221ebc00-7949-4e38-b480-a583cee36f24" containerName="payload-pre-processing"
Sep 09 21:20:19.780761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780616    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="221ebc00-7949-4e38-b480-a583cee36f24" containerName="payload-pre-processing"
Sep 09 21:20:19.780761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780683    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e275d100-7e42-4f3b-a7e8-1c8c7d8e0cdf" containerName="main"
Sep 09 21:20:19.780761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780692    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="221ebc00-7949-4e38-b480-a583cee36f24" containerName="payload-pre-processing"
Sep 09 21:20:19.780761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780701    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ef5458-58b0-412f-ba21-0d30e1f05310" containerName="revoke-keys"
Sep 09 21:20:19.780761 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.780706    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="16cf2cb6-3931-4dd9-b2a1-de5b5e72b1bc" containerName="payload-processing"
Sep 09 21:20:19.783809 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.783795    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"
Sep 09 21:20:19.786788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.786766    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:20:19.796335 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.796309    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"]
Sep 09 21:20:19.877583 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.877539    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3706a5f8-d21a-42f3-99af-f891e7d809f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5\" (UID: \"3706a5f8-d21a-42f3-99af-f891e7d809f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"
Sep 09 21:20:19.923253 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.923208    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:19.978458 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.978419    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3706a5f8-d21a-42f3-99af-f891e7d809f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5\" (UID: \"3706a5f8-d21a-42f3-99af-f891e7d809f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"
Sep 09 21:20:19.979072 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:19.979051    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3706a5f8-d21a-42f3-99af-f891e7d809f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5\" (UID: \"3706a5f8-d21a-42f3-99af-f891e7d809f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"
Sep 09 21:20:20.095926 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:20.095825    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"
Sep 09 21:20:20.382011 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:20.381985    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"]
Sep 09 21:20:20.383707 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:20:20.383676    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3706a5f8_d21a_42f3_99af_f891e7d809f5.slice/crio-0795ac9ae71cbef90a3699f3dcb93d82a9e57971231434b69f9777e04f8e7ea7 WatchSource:0}: Error finding container 0795ac9ae71cbef90a3699f3dcb93d82a9e57971231434b69f9777e04f8e7ea7: Status 404 returned error can't find the container with id 0795ac9ae71cbef90a3699f3dcb93d82a9e57971231434b69f9777e04f8e7ea7
Sep 09 21:20:20.779412 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:20.779374    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5" event={"ID":"3706a5f8-d21a-42f3-99af-f891e7d809f5","Type":"ContainerStarted","Data":"f36ee9465c078e156845a101d1108b844926a4d4dae8348fb54dfc0aedcd9628"}
Sep 09 21:20:20.779615 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:20.779425    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5" event={"ID":"3706a5f8-d21a-42f3-99af-f891e7d809f5","Type":"ContainerStarted","Data":"0795ac9ae71cbef90a3699f3dcb93d82a9e57971231434b69f9777e04f8e7ea7"}
Sep 09 21:20:20.798511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:20.798455    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5" podStartSLOduration=1.798439098 podStartE2EDuration="1.798439098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:20:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:20:20.796617628 +0000 UTC m=+1914.767184345" watchObservedRunningTime="2026-09-09 21:20:20.798439098 +0000 UTC m=+1914.769005871"
Sep 09 21:20:21.787275 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:21.787244    2571 generic.go:356] "Generic (PLEG): container finished" podID="3706a5f8-d21a-42f3-99af-f891e7d809f5" containerID="f36ee9465c078e156845a101d1108b844926a4d4dae8348fb54dfc0aedcd9628" exitCode=0
Sep 09 21:20:21.787685 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:21.787338    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5" event={"ID":"3706a5f8-d21a-42f3-99af-f891e7d809f5","Type":"ContainerDied","Data":"f36ee9465c078e156845a101d1108b844926a4d4dae8348fb54dfc0aedcd9628"}
Sep 09 21:20:22.936493 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:22.936468    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"
Sep 09 21:20:23.110754 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:23.110659    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3706a5f8-d21a-42f3-99af-f891e7d809f5-maas-api-service-ca\") pod \"3706a5f8-d21a-42f3-99af-f891e7d809f5\" (UID: \"3706a5f8-d21a-42f3-99af-f891e7d809f5\") "
Sep 09 21:20:23.111047 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:23.111025    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3706a5f8-d21a-42f3-99af-f891e7d809f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3706a5f8-d21a-42f3-99af-f891e7d809f5" (UID: "3706a5f8-d21a-42f3-99af-f891e7d809f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:20:23.212317 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:23.212285    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3706a5f8-d21a-42f3-99af-f891e7d809f5-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:20:23.800816 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:23.800789    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"
Sep 09 21:20:23.801090 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:23.800817    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5" event={"ID":"3706a5f8-d21a-42f3-99af-f891e7d809f5","Type":"ContainerDied","Data":"0795ac9ae71cbef90a3699f3dcb93d82a9e57971231434b69f9777e04f8e7ea7"}
Sep 09 21:20:23.801090 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:23.800858    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0795ac9ae71cbef90a3699f3dcb93d82a9e57971231434b69f9777e04f8e7ea7"
Sep 09 21:20:30.437407 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.437368    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:30.453253 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.453212    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"]
Sep 09 21:20:30.453500 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.453469    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j" podUID="40b69aae-caeb-4d99-8cb2-6c2b77ca15e2" containerName="payload-processing" containerID="cri-o://e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173" gracePeriod=30
Sep 09 21:20:30.512399 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.512367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:30.668966 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.668908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:30.689403 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.689317    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"]
Sep 09 21:20:30.693915 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.693883    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-d7dd56-nntd5"]
Sep 09 21:20:30.720710 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.720684    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:20:30.747229 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.747193    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:30.787543 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.787507    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptq97\" (UniqueName: \"kubernetes.io/projected/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-kube-api-access-ptq97\") pod \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\" (UID: \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\") "
Sep 09 21:20:30.787717 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.787567    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-plugins-config-volume\") pod \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\" (UID: \"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2\") "
Sep 09 21:20:30.787947 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.787925    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40b69aae-caeb-4d99-8cb2-6c2b77ca15e2" (UID: "40b69aae-caeb-4d99-8cb2-6c2b77ca15e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:20:30.789780 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.789755    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-kube-api-access-ptq97" (OuterVolumeSpecName: "kube-api-access-ptq97") pod "40b69aae-caeb-4d99-8cb2-6c2b77ca15e2" (UID: "40b69aae-caeb-4d99-8cb2-6c2b77ca15e2"). InnerVolumeSpecName "kube-api-access-ptq97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:20:30.848632 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.848598    2571 generic.go:356] "Generic (PLEG): container finished" podID="40b69aae-caeb-4d99-8cb2-6c2b77ca15e2" containerID="e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173" exitCode=0
Sep 09 21:20:30.848797 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.848641    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"
Sep 09 21:20:30.848797 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.848650    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j" event={"ID":"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2","Type":"ContainerDied","Data":"e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173"}
Sep 09 21:20:30.848797 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.848700    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j" event={"ID":"40b69aae-caeb-4d99-8cb2-6c2b77ca15e2","Type":"ContainerDied","Data":"2bbadc10919090c1a2b6994ad636780684446f74575917e39486606e0325f414"}
Sep 09 21:20:30.848797 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.848724    2571 scope.go:117] "RemoveContainer" containerID="e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173"
Sep 09 21:20:30.859353 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.859334    2571 scope.go:117] "RemoveContainer" containerID="e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173"
Sep 09 21:20:30.859625 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:20:30.859604    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173\": container with ID starting with e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173 not found: ID does not exist" containerID="e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173"
Sep 09 21:20:30.859686 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.859639    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173"} err="failed to get container status \"e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173\": rpc error: code = NotFound desc = could not find container \"e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173\": container with ID starting with e925172ce094b2d196f3b110703d631aa1ef5d79fddd4a862c09bf6cfec79173 not found: ID does not exist"
Sep 09 21:20:30.880471 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.880446    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"]
Sep 09 21:20:30.886130 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.886111    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-d7dd56-656bbffd75-t5p9j"]
Sep 09 21:20:30.888516 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.888496    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptq97\" (UniqueName: \"kubernetes.io/projected/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-kube-api-access-ptq97\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:20:30.888596 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:30.888520    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:20:31.101900 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:31.101858    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:32.566239 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:32.566196    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3706a5f8-d21a-42f3-99af-f891e7d809f5" path="/var/lib/kubelet/pods/3706a5f8-d21a-42f3-99af-f891e7d809f5/volumes"
Sep 09 21:20:32.566776 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:32.566756    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40b69aae-caeb-4d99-8cb2-6c2b77ca15e2" path="/var/lib/kubelet/pods/40b69aae-caeb-4d99-8cb2-6c2b77ca15e2/volumes"
Sep 09 21:20:33.852862 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:33.852821    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:37.384096 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:37.384050    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:37.484244 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:37.484196    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:37.691183 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:37.691056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:38.020424 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.020388    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"]
Sep 09 21:20:38.020823 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.020810    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40b69aae-caeb-4d99-8cb2-6c2b77ca15e2" containerName="payload-processing"
Sep 09 21:20:38.020884 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.020824    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="40b69aae-caeb-4d99-8cb2-6c2b77ca15e2" containerName="payload-processing"
Sep 09 21:20:38.020884 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.020846    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3706a5f8-d21a-42f3-99af-f891e7d809f5" containerName="revoke-keys"
Sep 09 21:20:38.020884 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.020851    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3706a5f8-d21a-42f3-99af-f891e7d809f5" containerName="revoke-keys"
Sep 09 21:20:38.020982 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.020910    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3706a5f8-d21a-42f3-99af-f891e7d809f5" containerName="revoke-keys"
Sep 09 21:20:38.020982 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.020920    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="40b69aae-caeb-4d99-8cb2-6c2b77ca15e2" containerName="payload-processing"
Sep 09 21:20:38.025394 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.025371    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"
Sep 09 21:20:38.027921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.027904    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:20:38.035067 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.035035    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"]
Sep 09 21:20:38.166093 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.166048    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ea3b15-a779-466c-9733-50b795ef3962-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt\" (UID: \"a0ea3b15-a779-466c-9733-50b795ef3962\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"
Sep 09 21:20:38.267622 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.267581    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ea3b15-a779-466c-9733-50b795ef3962-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt\" (UID: \"a0ea3b15-a779-466c-9733-50b795ef3962\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"
Sep 09 21:20:38.268212 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.268195    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ea3b15-a779-466c-9733-50b795ef3962-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt\" (UID: \"a0ea3b15-a779-466c-9733-50b795ef3962\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"
Sep 09 21:20:38.277269 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.277190    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:38.337218 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.337183    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"
Sep 09 21:20:38.617624 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.617596    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"]
Sep 09 21:20:38.619494 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:20:38.619469    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0ea3b15_a779_466c_9733_50b795ef3962.slice/crio-4d0d4c297b5554f1ee4bc758f62b32caa77be36022c47144f7de709d302e284e WatchSource:0}: Error finding container 4d0d4c297b5554f1ee4bc758f62b32caa77be36022c47144f7de709d302e284e: Status 404 returned error can't find the container with id 4d0d4c297b5554f1ee4bc758f62b32caa77be36022c47144f7de709d302e284e
Sep 09 21:20:38.904701 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:38.904663    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt" event={"ID":"a0ea3b15-a779-466c-9733-50b795ef3962","Type":"ContainerStarted","Data":"4d0d4c297b5554f1ee4bc758f62b32caa77be36022c47144f7de709d302e284e"}
Sep 09 21:20:39.913390 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:39.913359    2571 generic.go:356] "Generic (PLEG): container finished" podID="a0ea3b15-a779-466c-9733-50b795ef3962" containerID="bb57adaa0859c5845c70503dacdd97cc0449ff5ff096371810442f0d3b426bf9" exitCode=0
Sep 09 21:20:39.913838 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:39.913413    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt" event={"ID":"a0ea3b15-a779-466c-9733-50b795ef3962","Type":"ContainerDied","Data":"bb57adaa0859c5845c70503dacdd97cc0449ff5ff096371810442f0d3b426bf9"}
Sep 09 21:20:41.052926 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:41.052896    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"
Sep 09 21:20:41.199396 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:41.199361    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ea3b15-a779-466c-9733-50b795ef3962-maas-api-service-ca\") pod \"a0ea3b15-a779-466c-9733-50b795ef3962\" (UID: \"a0ea3b15-a779-466c-9733-50b795ef3962\") "
Sep 09 21:20:41.199720 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:41.199695    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0ea3b15-a779-466c-9733-50b795ef3962-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a0ea3b15-a779-466c-9733-50b795ef3962" (UID: "a0ea3b15-a779-466c-9733-50b795ef3962"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:20:41.300792 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:41.300756    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a0ea3b15-a779-466c-9733-50b795ef3962-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:20:41.926691 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:41.926661    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"
Sep 09 21:20:41.926880 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:41.926707    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt" event={"ID":"a0ea3b15-a779-466c-9733-50b795ef3962","Type":"ContainerDied","Data":"4d0d4c297b5554f1ee4bc758f62b32caa77be36022c47144f7de709d302e284e"}
Sep 09 21:20:41.926880 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:41.926759    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d0d4c297b5554f1ee4bc758f62b32caa77be36022c47144f7de709d302e284e"
Sep 09 21:20:48.641730 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.641699    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"]
Sep 09 21:20:48.644666 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.644638    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-3d3cac-29816475-4gzlw"]
Sep 09 21:20:48.646573 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.646549    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:48.697970 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.697938    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"]
Sep 09 21:20:48.698235 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.698184    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c" podUID="6a6f0b37-4214-499a-aa6b-54859916bca9" containerName="payload-pre-processing" containerID="cri-o://878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5" gracePeriod=30
Sep 09 21:20:48.772975 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.772930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:48.882042 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.882004    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:48.954686 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.954665    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:20:48.975197 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.975167    2571 generic.go:356] "Generic (PLEG): container finished" podID="6a6f0b37-4214-499a-aa6b-54859916bca9" containerID="878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5" exitCode=0
Sep 09 21:20:48.975316 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.975207    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"
Sep 09 21:20:48.975316 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.975254    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c" event={"ID":"6a6f0b37-4214-499a-aa6b-54859916bca9","Type":"ContainerDied","Data":"878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5"}
Sep 09 21:20:48.975316 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.975302    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c" event={"ID":"6a6f0b37-4214-499a-aa6b-54859916bca9","Type":"ContainerDied","Data":"6841b3f2dca2254e49e4ead0dbeba2ac5981667d520bd5f435a54500e0425cef"}
Sep 09 21:20:48.975434 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.975325    2571 scope.go:117] "RemoveContainer" containerID="878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5"
Sep 09 21:20:48.987879 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.987661    2571 scope.go:117] "RemoveContainer" containerID="878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5"
Sep 09 21:20:48.990351 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:20:48.990317    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5\": container with ID starting with 878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5 not found: ID does not exist" containerID="878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5"
Sep 09 21:20:48.990472 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.990365    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5"} err="failed to get container status \"878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5\": rpc error: code = NotFound desc = could not find container \"878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5\": container with ID starting with 878381b07368af51be09e56fe65c4a80eeddfb19e7a5e43fe120d310507adaa5 not found: ID does not exist"
Sep 09 21:20:48.996519 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:48.996490    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"]
Sep 09 21:20:49.005042 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.005012    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-3d3cac-k8gjt"]
Sep 09 21:20:49.011788 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.011756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:49.074811 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.074777    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4x62\" (UniqueName: \"kubernetes.io/projected/6a6f0b37-4214-499a-aa6b-54859916bca9-kube-api-access-b4x62\") pod \"6a6f0b37-4214-499a-aa6b-54859916bca9\" (UID: \"6a6f0b37-4214-499a-aa6b-54859916bca9\") "
Sep 09 21:20:49.075009 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.074916    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a6f0b37-4214-499a-aa6b-54859916bca9-plugins-config-volume\") pod \"6a6f0b37-4214-499a-aa6b-54859916bca9\" (UID: \"6a6f0b37-4214-499a-aa6b-54859916bca9\") "
Sep 09 21:20:49.075313 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.075290    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a6f0b37-4214-499a-aa6b-54859916bca9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6a6f0b37-4214-499a-aa6b-54859916bca9" (UID: "6a6f0b37-4214-499a-aa6b-54859916bca9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:20:49.077058 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.077038    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a6f0b37-4214-499a-aa6b-54859916bca9-kube-api-access-b4x62" (OuterVolumeSpecName: "kube-api-access-b4x62") pod "6a6f0b37-4214-499a-aa6b-54859916bca9" (UID: "6a6f0b37-4214-499a-aa6b-54859916bca9"). InnerVolumeSpecName "kube-api-access-b4x62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:20:49.147324 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.147292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:49.176433 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.176399    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4x62\" (UniqueName: \"kubernetes.io/projected/6a6f0b37-4214-499a-aa6b-54859916bca9-kube-api-access-b4x62\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:20:49.176433 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.176431    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a6f0b37-4214-499a-aa6b-54859916bca9-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:20:49.300912 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.300881    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"]
Sep 09 21:20:49.305315 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.305288    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3d3cac-5755d69fcd-n5j2c"]
Sep 09 21:20:49.396560 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:49.396520    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:50.567180 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:50.567149    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ec01e37-859b-4f37-b3fa-e4640985a0c5" path="/var/lib/kubelet/pods/5ec01e37-859b-4f37-b3fa-e4640985a0c5/volumes"
Sep 09 21:20:50.567634 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:50.567619    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a6f0b37-4214-499a-aa6b-54859916bca9" path="/var/lib/kubelet/pods/6a6f0b37-4214-499a-aa6b-54859916bca9/volumes"
Sep 09 21:20:50.567945 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:50.567932    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0ea3b15-a779-466c-9733-50b795ef3962" path="/var/lib/kubelet/pods/a0ea3b15-a779-466c-9733-50b795ef3962/volumes"
Sep 09 21:20:56.430822 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:56.430783    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:56.590452 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:56.590417    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:56.767953 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:56.767915    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:57.087424 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.087341    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"]
Sep 09 21:20:57.087778 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.087765    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ea3b15-a779-466c-9733-50b795ef3962" containerName="revoke-keys"
Sep 09 21:20:57.087824 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.087779    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ea3b15-a779-466c-9733-50b795ef3962" containerName="revoke-keys"
Sep 09 21:20:57.087824 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.087798    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a6f0b37-4214-499a-aa6b-54859916bca9" containerName="payload-pre-processing"
Sep 09 21:20:57.087824 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.087804    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a6f0b37-4214-499a-aa6b-54859916bca9" containerName="payload-pre-processing"
Sep 09 21:20:57.087922 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.087870    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ea3b15-a779-466c-9733-50b795ef3962" containerName="revoke-keys"
Sep 09 21:20:57.087922 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.087879    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a6f0b37-4214-499a-aa6b-54859916bca9" containerName="payload-pre-processing"
Sep 09 21:20:57.092548 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.092528    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"
Sep 09 21:20:57.096820 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.096799    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6l2wq\""
Sep 09 21:20:57.111742 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.111714    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"]
Sep 09 21:20:57.160939 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.160907    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9b06b-9e92-4976-b25f-3852ba2050b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224\" (UID: \"6ea9b06b-9e92-4976-b25f-3852ba2050b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"
Sep 09 21:20:57.261362 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.261331    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9b06b-9e92-4976-b25f-3852ba2050b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224\" (UID: \"6ea9b06b-9e92-4976-b25f-3852ba2050b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"
Sep 09 21:20:57.261970 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.261952    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9b06b-9e92-4976-b25f-3852ba2050b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224\" (UID: \"6ea9b06b-9e92-4976-b25f-3852ba2050b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"
Sep 09 21:20:57.293880 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.293846    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:20:57.402890 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.402798    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"
Sep 09 21:20:57.690579 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:57.690549    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"]
Sep 09 21:20:57.692214 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:20:57.692177    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea9b06b_9e92_4976_b25f_3852ba2050b1.slice/crio-2e829c78d5decc59630faaf70a840d3263a085531fe7cfdbf33864480680539e WatchSource:0}: Error finding container 2e829c78d5decc59630faaf70a840d3263a085531fe7cfdbf33864480680539e: Status 404 returned error can't find the container with id 2e829c78d5decc59630faaf70a840d3263a085531fe7cfdbf33864480680539e
Sep 09 21:20:58.036145 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:58.036111    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224" event={"ID":"6ea9b06b-9e92-4976-b25f-3852ba2050b1","Type":"ContainerStarted","Data":"3112dee768d64ad399f9082519a8728f84d96634c8cd164228a027c8c803fd06"}
Sep 09 21:20:58.036319 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:58.036163    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224" event={"ID":"6ea9b06b-9e92-4976-b25f-3852ba2050b1","Type":"ContainerStarted","Data":"2e829c78d5decc59630faaf70a840d3263a085531fe7cfdbf33864480680539e"}
Sep 09 21:20:58.082395 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:58.082342    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224" podStartSLOduration=1.0823264400000001 podStartE2EDuration="1.08232644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:20:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:20:58.080576508 +0000 UTC m=+1952.051143222" watchObservedRunningTime="2026-09-09 21:20:58.08232644 +0000 UTC m=+1952.052893156"
Sep 09 21:20:59.043941 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:59.043907    2571 generic.go:356] "Generic (PLEG): container finished" podID="6ea9b06b-9e92-4976-b25f-3852ba2050b1" containerID="3112dee768d64ad399f9082519a8728f84d96634c8cd164228a027c8c803fd06" exitCode=0
Sep 09 21:20:59.044350 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:20:59.043996    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224" event={"ID":"6ea9b06b-9e92-4976-b25f-3852ba2050b1","Type":"ContainerDied","Data":"3112dee768d64ad399f9082519a8728f84d96634c8cd164228a027c8c803fd06"}
Sep 09 21:21:00.188099 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:00.188049    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"
Sep 09 21:21:00.285000 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:00.284966    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9b06b-9e92-4976-b25f-3852ba2050b1-maas-api-service-ca\") pod \"6ea9b06b-9e92-4976-b25f-3852ba2050b1\" (UID: \"6ea9b06b-9e92-4976-b25f-3852ba2050b1\") "
Sep 09 21:21:00.285271 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:00.285252    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea9b06b-9e92-4976-b25f-3852ba2050b1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ea9b06b-9e92-4976-b25f-3852ba2050b1" (UID: "6ea9b06b-9e92-4976-b25f-3852ba2050b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:21:00.385966 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:00.385875    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9b06b-9e92-4976-b25f-3852ba2050b1-maas-api-service-ca\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:21:01.058292 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:01.058262    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"
Sep 09 21:21:01.058478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:01.058295    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224" event={"ID":"6ea9b06b-9e92-4976-b25f-3852ba2050b1","Type":"ContainerDied","Data":"2e829c78d5decc59630faaf70a840d3263a085531fe7cfdbf33864480680539e"}
Sep 09 21:21:01.058478 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:01.058338    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e829c78d5decc59630faaf70a840d3263a085531fe7cfdbf33864480680539e"
Sep 09 21:21:07.741850 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:07.741815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:07.778486 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:07.778448    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"]
Sep 09 21:21:07.778706 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:07.778669    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c" podUID="a12a51d9-9437-4591-b3a3-3593ac78e5de" containerName="payload-processing" containerID="cri-o://7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb" gracePeriod=30
Sep 09 21:21:07.788599 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:07.788571    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"]
Sep 09 21:21:07.789392 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:07.788793    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5" podUID="3e9ed0c9-46ea-4cab-8895-53ba26c45254" containerName="payload-pre-processing" containerID="cri-o://7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c" gracePeriod=30
Sep 09 21:21:07.863944 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:07.863909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:08.056162 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.056128    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:08.065993 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.065972    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:21:08.069176 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.069154    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:21:08.114849 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.114818    2571 generic.go:356] "Generic (PLEG): container finished" podID="3e9ed0c9-46ea-4cab-8895-53ba26c45254" containerID="7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c" exitCode=0
Sep 09 21:21:08.115042 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.114861    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"
Sep 09 21:21:08.115042 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.114864    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5" event={"ID":"3e9ed0c9-46ea-4cab-8895-53ba26c45254","Type":"ContainerDied","Data":"7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c"}
Sep 09 21:21:08.115042 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.114918    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5" event={"ID":"3e9ed0c9-46ea-4cab-8895-53ba26c45254","Type":"ContainerDied","Data":"f42fdd43706876e2745bc89dc481611bff9b703d69095fa210fd9a3db47d2f22"}
Sep 09 21:21:08.115042 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.114944    2571 scope.go:117] "RemoveContainer" containerID="7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c"
Sep 09 21:21:08.116454 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.116428    2571 generic.go:356] "Generic (PLEG): container finished" podID="a12a51d9-9437-4591-b3a3-3593ac78e5de" containerID="7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb" exitCode=0
Sep 09 21:21:08.116561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.116460    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c" event={"ID":"a12a51d9-9437-4591-b3a3-3593ac78e5de","Type":"ContainerDied","Data":"7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb"}
Sep 09 21:21:08.116561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.116481    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"
Sep 09 21:21:08.116561 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.116488    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c" event={"ID":"a12a51d9-9437-4591-b3a3-3593ac78e5de","Type":"ContainerDied","Data":"70083ec71f36bf4bf685094c5bdab5ebf41e07058253eade6570a452a5721734"}
Sep 09 21:21:08.126424 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.126400    2571 scope.go:117] "RemoveContainer" containerID="7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c"
Sep 09 21:21:08.126770 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:21:08.126750    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c\": container with ID starting with 7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c not found: ID does not exist" containerID="7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c"
Sep 09 21:21:08.126831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.126780    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c"} err="failed to get container status \"7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c\": rpc error: code = NotFound desc = could not find container \"7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c\": container with ID starting with 7ec3de67e218401a763f40949c27b29a32433097aeff314ca5d5a685b4c6c59c not found: ID does not exist"
Sep 09 21:21:08.126831 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.126798    2571 scope.go:117] "RemoveContainer" containerID="7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb"
Sep 09 21:21:08.138330 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.138179    2571 scope.go:117] "RemoveContainer" containerID="7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb"
Sep 09 21:21:08.138821 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:21:08.138786    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb\": container with ID starting with 7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb not found: ID does not exist" containerID="7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb"
Sep 09 21:21:08.138969 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.138831    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb"} err="failed to get container status \"7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb\": rpc error: code = NotFound desc = could not find container \"7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb\": container with ID starting with 7f297ded27dd430edb4b5c2892de0c2c4a292511f70c981f30161d39d26495cb not found: ID does not exist"
Sep 09 21:21:08.139773 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.139746    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"]
Sep 09 21:21:08.147776 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.147750    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-8db5a7-l8224"]
Sep 09 21:21:08.157347 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.157317    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9ed0c9-46ea-4cab-8895-53ba26c45254-plugins-config-volume\") pod \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\" (UID: \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\") "
Sep 09 21:21:08.157483 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.157371    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12a51d9-9437-4591-b3a3-3593ac78e5de-plugins-config-volume\") pod \"a12a51d9-9437-4591-b3a3-3593ac78e5de\" (UID: \"a12a51d9-9437-4591-b3a3-3593ac78e5de\") "
Sep 09 21:21:08.157483 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.157409    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llpfm\" (UniqueName: \"kubernetes.io/projected/3e9ed0c9-46ea-4cab-8895-53ba26c45254-kube-api-access-llpfm\") pod \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\" (UID: \"3e9ed0c9-46ea-4cab-8895-53ba26c45254\") "
Sep 09 21:21:08.157483 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.157460    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4jb8\" (UniqueName: \"kubernetes.io/projected/a12a51d9-9437-4591-b3a3-3593ac78e5de-kube-api-access-k4jb8\") pod \"a12a51d9-9437-4591-b3a3-3593ac78e5de\" (UID: \"a12a51d9-9437-4591-b3a3-3593ac78e5de\") "
Sep 09 21:21:08.157742 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.157719    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e9ed0c9-46ea-4cab-8895-53ba26c45254-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e9ed0c9-46ea-4cab-8895-53ba26c45254" (UID: "3e9ed0c9-46ea-4cab-8895-53ba26c45254"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:21:08.157852 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.157829    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a12a51d9-9437-4591-b3a3-3593ac78e5de-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a12a51d9-9437-4591-b3a3-3593ac78e5de" (UID: "a12a51d9-9437-4591-b3a3-3593ac78e5de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 21:21:08.159840 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.159819    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e9ed0c9-46ea-4cab-8895-53ba26c45254-kube-api-access-llpfm" (OuterVolumeSpecName: "kube-api-access-llpfm") pod "3e9ed0c9-46ea-4cab-8895-53ba26c45254" (UID: "3e9ed0c9-46ea-4cab-8895-53ba26c45254"). InnerVolumeSpecName "kube-api-access-llpfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:21:08.159925 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.159857    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a12a51d9-9437-4591-b3a3-3593ac78e5de-kube-api-access-k4jb8" (OuterVolumeSpecName: "kube-api-access-k4jb8") pod "a12a51d9-9437-4591-b3a3-3593ac78e5de" (UID: "a12a51d9-9437-4591-b3a3-3593ac78e5de"). InnerVolumeSpecName "kube-api-access-k4jb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:21:08.205023 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.204984    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:08.259059 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.259022    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e9ed0c9-46ea-4cab-8895-53ba26c45254-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:21:08.259059 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.259053    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a12a51d9-9437-4591-b3a3-3593ac78e5de-plugins-config-volume\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:21:08.259059 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.259062    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llpfm\" (UniqueName: \"kubernetes.io/projected/3e9ed0c9-46ea-4cab-8895-53ba26c45254-kube-api-access-llpfm\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:21:08.259316 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.259070    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4jb8\" (UniqueName: \"kubernetes.io/projected/a12a51d9-9437-4591-b3a3-3593ac78e5de-kube-api-access-k4jb8\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:21:08.445511 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.445481    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"]
Sep 09 21:21:08.464015 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.463984    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8db5a7-545f7ffdc4-tmzt5"]
Sep 09 21:21:08.479694 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.479667    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"]
Sep 09 21:21:08.483326 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.483300    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-8db5a7-db7789959-cf68c"]
Sep 09 21:21:08.528687 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.528657    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:08.566842 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.566808    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e9ed0c9-46ea-4cab-8895-53ba26c45254" path="/var/lib/kubelet/pods/3e9ed0c9-46ea-4cab-8895-53ba26c45254/volumes"
Sep 09 21:21:08.567169 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.567155    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ea9b06b-9e92-4976-b25f-3852ba2050b1" path="/var/lib/kubelet/pods/6ea9b06b-9e92-4976-b25f-3852ba2050b1/volumes"
Sep 09 21:21:08.567475 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:08.567463    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a12a51d9-9437-4591-b3a3-3593ac78e5de" path="/var/lib/kubelet/pods/a12a51d9-9437-4591-b3a3-3593ac78e5de/volumes"
Sep 09 21:21:15.105273 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:15.105209    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:15.281204 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:15.281170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:15.495938 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:15.495909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:21:27.041985 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:27.041902    2571 scope.go:117] "RemoveContainer" containerID="3d9138e31273a9942330cc52ad194d9a1925bf4eaac907391d55e6423afbe5ac"
Sep 09 21:21:27.051545 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:27.051524    2571 scope.go:117] "RemoveContainer" containerID="00a4f46292931e480341c1339da5c3f64c6e3c4fc534b08d27b5eb8412799d57"
Sep 09 21:21:27.060570 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:27.060551    2571 scope.go:117] "RemoveContainer" containerID="e7ac582a164252f02365372da5dc9f34df0b8d07e9c65b1b4df87fd3d1bc7aed"
Sep 09 21:21:27.069151 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:27.069129    2571 scope.go:117] "RemoveContainer" containerID="5aa6a98a1016d73d0bb1aeeb1715b2560cc474ce6e7a35aeb0f4533463438b23"
Sep 09 21:21:27.077604 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:21:27.077588    2571 scope.go:117] "RemoveContainer" containerID="b9c40f471b0b38bcb55a8b12085c9a98381c6e0cf77b61282e99341f2019a2ab"
Sep 09 21:22:19.836186 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:19.836143    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:19.935345 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:19.935313    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:27.139980 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:27.139944    2571 scope.go:117] "RemoveContainer" containerID="cd634dfb70ffcb8b3d536013693a67a3ee50e3b6756b9528c333e93ad1d9f5fc"
Sep 09 21:22:27.149538 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:27.149516    2571 scope.go:117] "RemoveContainer" containerID="81ba05fac37995bc23343b08aa9d58211dc53e17aa2059bf8ae054fdff5b6a10"
Sep 09 21:22:27.159374 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:27.159352    2571 scope.go:117] "RemoveContainer" containerID="8ce3b870198c46f454eb4c651e927ab339ce4b4148a2778855b3be37c17f08d2"
Sep 09 21:22:42.154043 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:42.154014    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:43.328122 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:43.328064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:44.803351 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:44.803291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:49.760208 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:49.760122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:22:54.776527 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:22:54.776470    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:22.994605 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:23:22.994570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:23:23.127362 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:23:23.127314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:24:05.875901 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:24:05.875859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:27.215133 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:27.215094    2571 scope.go:117] "RemoveContainer" containerID="a891fcab2b757170cafa8110f0603372cd22b133e04d2bb9b6b9663c55cd75cb"
Sep 09 21:25:27.231605 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:27.231557    2571 scope.go:117] "RemoveContainer" containerID="740217bad8aa8394160c782dc44030c1eab6531a5cbd0deb1e68d38ab54c8b09"
Sep 09 21:25:32.018294 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018263    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"]
Sep 09 21:25:32.018726 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018712    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a12a51d9-9437-4591-b3a3-3593ac78e5de" containerName="payload-processing"
Sep 09 21:25:32.018772 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018728    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a12a51d9-9437-4591-b3a3-3593ac78e5de" containerName="payload-processing"
Sep 09 21:25:32.018772 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018742    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e9ed0c9-46ea-4cab-8895-53ba26c45254" containerName="payload-pre-processing"
Sep 09 21:25:32.018772 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018748    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e9ed0c9-46ea-4cab-8895-53ba26c45254" containerName="payload-pre-processing"
Sep 09 21:25:32.018772 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018759    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ea9b06b-9e92-4976-b25f-3852ba2050b1" containerName="revoke-keys"
Sep 09 21:25:32.018772 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018764    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ea9b06b-9e92-4976-b25f-3852ba2050b1" containerName="revoke-keys"
Sep 09 21:25:32.018941 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018830    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ea9b06b-9e92-4976-b25f-3852ba2050b1" containerName="revoke-keys"
Sep 09 21:25:32.018941 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018841    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a12a51d9-9437-4591-b3a3-3593ac78e5de" containerName="payload-processing"
Sep 09 21:25:32.018941 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.018850    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e9ed0c9-46ea-4cab-8895-53ba26c45254" containerName="payload-pre-processing"
Sep 09 21:25:32.021990 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.021973    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.024465 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.024441    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-694fed88-kserve-self-signed-certs\""
Sep 09 21:25:32.030643 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.030617    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"]
Sep 09 21:25:32.062302 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.062239    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:32.124449 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.124421    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-tmp-dir\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.124607 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.124455    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-model-cache\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.124607 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.124486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2v2r\" (UniqueName: \"kubernetes.io/projected/1fe12070-0583-4b9a-a73f-56edefa17d90-kube-api-access-n2v2r\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.124607 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.124539    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fe12070-0583-4b9a-a73f-56edefa17d90-tls-certs\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.124607 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.124568    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-home\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.124767 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.124674    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-dshm\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.225337 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225304    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fe12070-0583-4b9a-a73f-56edefa17d90-tls-certs\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.225473 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225346    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-home\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.225543 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225524    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-dshm\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.225609 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225592    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-tmp-dir\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.225643 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225628    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-model-cache\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.225677 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225657    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2v2r\" (UniqueName: \"kubernetes.io/projected/1fe12070-0583-4b9a-a73f-56edefa17d90-kube-api-access-n2v2r\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.225729 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225708    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-home\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.225929 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225909    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-tmp-dir\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.226043 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.225951    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-model-cache\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.227849 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.227829    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-dshm\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.228017 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.227999    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fe12070-0583-4b9a-a73f-56edefa17d90-tls-certs\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.235279 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.235253    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2v2r\" (UniqueName: \"kubernetes.io/projected/1fe12070-0583-4b9a-a73f-56edefa17d90-kube-api-access-n2v2r\") pod \"e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") " pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.334584 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.334467    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:32.609725 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.609692    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"]
Sep 09 21:25:32.611042 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:25:32.611014    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fe12070_0583_4b9a_a73f_56edefa17d90.slice/crio-86b10647c57e439c817444e6a92ea713b1c79d15e65759b61fbbf9237e369d0c WatchSource:0}: Error finding container 86b10647c57e439c817444e6a92ea713b1c79d15e65759b61fbbf9237e369d0c: Status 404 returned error can't find the container with id 86b10647c57e439c817444e6a92ea713b1c79d15e65759b61fbbf9237e369d0c
Sep 09 21:25:32.714032 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.713997    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x" event={"ID":"1fe12070-0583-4b9a-a73f-56edefa17d90","Type":"ContainerStarted","Data":"11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead"}
Sep 09 21:25:32.714210 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.714043    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x" event={"ID":"1fe12070-0583-4b9a-a73f-56edefa17d90","Type":"ContainerStarted","Data":"86b10647c57e439c817444e6a92ea713b1c79d15e65759b61fbbf9237e369d0c"}
Sep 09 21:25:32.733896 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:32.733844    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x" podStartSLOduration=1.733828428 podStartE2EDuration="1.733828428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:25:32.730431602 +0000 UTC m=+2226.700998320" watchObservedRunningTime="2026-09-09 21:25:32.733828428 +0000 UTC m=+2226.704395144"
Sep 09 21:25:42.334991 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:42.334952    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:42.335498 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:42.335028    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:42.347816 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:42.347794    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:42.783808 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:42.783778    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:25:49.908835 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.908738    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"]
Sep 09 21:25:49.912438 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.912417    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:49.914921 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.914903    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-694fed88-kserve-self-signed-certs\""
Sep 09 21:25:49.923226 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.923201    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"]
Sep 09 21:25:49.949941 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.949907    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:25:49.998975 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.998941    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6b6l\" (UniqueName: \"kubernetes.io/projected/d247881e-9dfa-4a80-afb1-3b1c96acea00-kube-api-access-b6b6l\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:49.998975 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.998975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-tmp-dir\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:49.999218 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.999009    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-dshm\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:49.999218 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.999091    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-home\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:49.999218 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.999127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d247881e-9dfa-4a80-afb1-3b1c96acea00-tls-certs\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:49.999218 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:49.999201    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-model-cache\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.099947 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.099905    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b6l\" (UniqueName: \"kubernetes.io/projected/d247881e-9dfa-4a80-afb1-3b1c96acea00-kube-api-access-b6b6l\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.099947 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.099945    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-tmp-dir\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.100251 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.100102    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-dshm\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.100251 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.100151    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-home\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.100251 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.100182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d247881e-9dfa-4a80-afb1-3b1c96acea00-tls-certs\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.100453 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.100250    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-model-cache\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.100453 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.100417    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-tmp-dir\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.100565 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.100490    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-home\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.100644 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.100621    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-model-cache\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.102562 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.102533    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-dshm\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.102768 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.102751    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d247881e-9dfa-4a80-afb1-3b1c96acea00-tls-certs\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.109802 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.109774    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6b6l\" (UniqueName: \"kubernetes.io/projected/d247881e-9dfa-4a80-afb1-3b1c96acea00-kube-api-access-b6b6l\") pod \"e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") " pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.224242 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.224213    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:25:50.500787 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.500759    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"]
Sep 09 21:25:50.501741 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:25:50.501710    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd247881e_9dfa_4a80_afb1_3b1c96acea00.slice/crio-351c74261e678422b216cbeee0cc431247f4a236bf462b13f8c80827a67a7622 WatchSource:0}: Error finding container 351c74261e678422b216cbeee0cc431247f4a236bf462b13f8c80827a67a7622: Status 404 returned error can't find the container with id 351c74261e678422b216cbeee0cc431247f4a236bf462b13f8c80827a67a7622
Sep 09 21:25:50.823053 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.822965    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx" event={"ID":"d247881e-9dfa-4a80-afb1-3b1c96acea00","Type":"ContainerStarted","Data":"27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71"}
Sep 09 21:25:50.823053 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.823002    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx" event={"ID":"d247881e-9dfa-4a80-afb1-3b1c96acea00","Type":"ContainerStarted","Data":"351c74261e678422b216cbeee0cc431247f4a236bf462b13f8c80827a67a7622"}
Sep 09 21:25:50.843252 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:25:50.843199    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx" podStartSLOduration=1.8431841329999998 podStartE2EDuration="1.843184133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:25:50.840927281 +0000 UTC m=+2244.811493999" watchObservedRunningTime="2026-09-09 21:25:50.843184133 +0000 UTC m=+2244.813750898"
Sep 09 21:26:00.224705 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:00.224665    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:26:00.225152 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:00.224746    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:26:00.237960 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:00.237934    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:26:00.892368 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:00.892340    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:26:09.063290 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:09.063256    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"]
Sep 09 21:26:09.090067 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:09.090030    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:10.923404 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:10.923369    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"]
Sep 09 21:26:10.951700 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:10.951653    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:26:24.083991 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.083919    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x" podUID="1fe12070-0583-4b9a-a73f-56edefa17d90" containerName="main" containerID="cri-o://11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead" gracePeriod=45
Sep 09 21:26:24.337647 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.337588    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:26:24.434704 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.434669    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-tmp-dir\") pod \"1fe12070-0583-4b9a-a73f-56edefa17d90\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") "
Sep 09 21:26:24.434704 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.434712    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fe12070-0583-4b9a-a73f-56edefa17d90-tls-certs\") pod \"1fe12070-0583-4b9a-a73f-56edefa17d90\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") "
Sep 09 21:26:24.434938 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.434748    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2v2r\" (UniqueName: \"kubernetes.io/projected/1fe12070-0583-4b9a-a73f-56edefa17d90-kube-api-access-n2v2r\") pod \"1fe12070-0583-4b9a-a73f-56edefa17d90\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") "
Sep 09 21:26:24.434938 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.434784    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-home\") pod \"1fe12070-0583-4b9a-a73f-56edefa17d90\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") "
Sep 09 21:26:24.435038 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.434941    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-dshm\") pod \"1fe12070-0583-4b9a-a73f-56edefa17d90\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") "
Sep 09 21:26:24.435038 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.435015    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-model-cache\") pod \"1fe12070-0583-4b9a-a73f-56edefa17d90\" (UID: \"1fe12070-0583-4b9a-a73f-56edefa17d90\") "
Sep 09 21:26:24.435038 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.435022    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1fe12070-0583-4b9a-a73f-56edefa17d90" (UID: "1fe12070-0583-4b9a-a73f-56edefa17d90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:26:24.435234 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.435035    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-home" (OuterVolumeSpecName: "home") pod "1fe12070-0583-4b9a-a73f-56edefa17d90" (UID: "1fe12070-0583-4b9a-a73f-56edefa17d90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:26:24.435298 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.435274    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-model-cache" (OuterVolumeSpecName: "model-cache") pod "1fe12070-0583-4b9a-a73f-56edefa17d90" (UID: "1fe12070-0583-4b9a-a73f-56edefa17d90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:26:24.435467 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.435452    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-model-cache\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:24.435510 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.435469    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-tmp-dir\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:24.435510 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.435477    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-home\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:24.437302 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.437270    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-dshm" (OuterVolumeSpecName: "dshm") pod "1fe12070-0583-4b9a-a73f-56edefa17d90" (UID: "1fe12070-0583-4b9a-a73f-56edefa17d90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:26:24.437555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.437534    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fe12070-0583-4b9a-a73f-56edefa17d90-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1fe12070-0583-4b9a-a73f-56edefa17d90" (UID: "1fe12070-0583-4b9a-a73f-56edefa17d90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:26:24.437632 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.437552    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fe12070-0583-4b9a-a73f-56edefa17d90-kube-api-access-n2v2r" (OuterVolumeSpecName: "kube-api-access-n2v2r") pod "1fe12070-0583-4b9a-a73f-56edefa17d90" (UID: "1fe12070-0583-4b9a-a73f-56edefa17d90"). InnerVolumeSpecName "kube-api-access-n2v2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:26:24.536883 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.536838    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fe12070-0583-4b9a-a73f-56edefa17d90-tls-certs\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:24.536883 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.536874    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2v2r\" (UniqueName: \"kubernetes.io/projected/1fe12070-0583-4b9a-a73f-56edefa17d90-kube-api-access-n2v2r\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:24.536883 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:24.536884    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fe12070-0583-4b9a-a73f-56edefa17d90-dshm\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:25.040260 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.040221    2571 generic.go:356] "Generic (PLEG): container finished" podID="1fe12070-0583-4b9a-a73f-56edefa17d90" containerID="11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead" exitCode=0
Sep 09 21:26:25.040260 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.040263    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"
Sep 09 21:26:25.040466 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.040315    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x" event={"ID":"1fe12070-0583-4b9a-a73f-56edefa17d90","Type":"ContainerDied","Data":"11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead"}
Sep 09 21:26:25.040466 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.040357    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x" event={"ID":"1fe12070-0583-4b9a-a73f-56edefa17d90","Type":"ContainerDied","Data":"86b10647c57e439c817444e6a92ea713b1c79d15e65759b61fbbf9237e369d0c"}
Sep 09 21:26:25.040466 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.040375    2571 scope.go:117] "RemoveContainer" containerID="11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead"
Sep 09 21:26:25.049967 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.049947    2571 scope.go:117] "RemoveContainer" containerID="11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead"
Sep 09 21:26:25.050353 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:26:25.050334    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead\": container with ID starting with 11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead not found: ID does not exist" containerID="11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead"
Sep 09 21:26:25.050431 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.050364    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead"} err="failed to get container status \"11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead\": rpc error: code = NotFound desc = could not find container \"11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead\": container with ID starting with 11026c74e4921899643b706e35a66dab4008eb1d3cb0facda73f6c24476d8ead not found: ID does not exist"
Sep 09 21:26:25.062288 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.062260    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"]
Sep 09 21:26:25.065889 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.065864    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-694fed88-kserve-7fc47976bc-9f75x"]
Sep 09 21:26:25.944484 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:25.944438    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx" podUID="d247881e-9dfa-4a80-afb1-3b1c96acea00" containerName="main" containerID="cri-o://27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71" gracePeriod=45
Sep 09 21:26:26.195064 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.195004    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:26:26.252779 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.252743    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-dshm\") pod \"d247881e-9dfa-4a80-afb1-3b1c96acea00\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") "
Sep 09 21:26:26.252974 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.252802    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d247881e-9dfa-4a80-afb1-3b1c96acea00-tls-certs\") pod \"d247881e-9dfa-4a80-afb1-3b1c96acea00\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") "
Sep 09 21:26:26.252974 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.252856    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-tmp-dir\") pod \"d247881e-9dfa-4a80-afb1-3b1c96acea00\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") "
Sep 09 21:26:26.252974 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.252877    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6b6l\" (UniqueName: \"kubernetes.io/projected/d247881e-9dfa-4a80-afb1-3b1c96acea00-kube-api-access-b6b6l\") pod \"d247881e-9dfa-4a80-afb1-3b1c96acea00\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") "
Sep 09 21:26:26.252974 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.252901    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-home\") pod \"d247881e-9dfa-4a80-afb1-3b1c96acea00\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") "
Sep 09 21:26:26.252974 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.252966    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-model-cache\") pod \"d247881e-9dfa-4a80-afb1-3b1c96acea00\" (UID: \"d247881e-9dfa-4a80-afb1-3b1c96acea00\") "
Sep 09 21:26:26.253306 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.253113    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d247881e-9dfa-4a80-afb1-3b1c96acea00" (UID: "d247881e-9dfa-4a80-afb1-3b1c96acea00"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:26:26.253370 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.253312    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-home" (OuterVolumeSpecName: "home") pod "d247881e-9dfa-4a80-afb1-3b1c96acea00" (UID: "d247881e-9dfa-4a80-afb1-3b1c96acea00"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:26:26.253370 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.253327    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-model-cache" (OuterVolumeSpecName: "model-cache") pod "d247881e-9dfa-4a80-afb1-3b1c96acea00" (UID: "d247881e-9dfa-4a80-afb1-3b1c96acea00"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:26:26.253488 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.253383    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-tmp-dir\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:26.255240 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.255218    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d247881e-9dfa-4a80-afb1-3b1c96acea00-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d247881e-9dfa-4a80-afb1-3b1c96acea00" (UID: "d247881e-9dfa-4a80-afb1-3b1c96acea00"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 21:26:26.255240 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.255216    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-dshm" (OuterVolumeSpecName: "dshm") pod "d247881e-9dfa-4a80-afb1-3b1c96acea00" (UID: "d247881e-9dfa-4a80-afb1-3b1c96acea00"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 21:26:26.255240 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.255216    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d247881e-9dfa-4a80-afb1-3b1c96acea00-kube-api-access-b6b6l" (OuterVolumeSpecName: "kube-api-access-b6b6l") pod "d247881e-9dfa-4a80-afb1-3b1c96acea00" (UID: "d247881e-9dfa-4a80-afb1-3b1c96acea00"). InnerVolumeSpecName "kube-api-access-b6b6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 21:26:26.354695 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.354655    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6b6l\" (UniqueName: \"kubernetes.io/projected/d247881e-9dfa-4a80-afb1-3b1c96acea00-kube-api-access-b6b6l\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:26.354695 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.354687    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-home\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:26.354695 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.354697    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-model-cache\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:26.354695 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.354707    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d247881e-9dfa-4a80-afb1-3b1c96acea00-dshm\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:26.354969 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.354715    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d247881e-9dfa-4a80-afb1-3b1c96acea00-tls-certs\") on node \"ip-10-0-129-13.ec2.internal\" DevicePath \"\""
Sep 09 21:26:26.566842 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:26.566763    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fe12070-0583-4b9a-a73f-56edefa17d90" path="/var/lib/kubelet/pods/1fe12070-0583-4b9a-a73f-56edefa17d90/volumes"
Sep 09 21:26:27.055256 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.055220    2571 generic.go:356] "Generic (PLEG): container finished" podID="d247881e-9dfa-4a80-afb1-3b1c96acea00" containerID="27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71" exitCode=0
Sep 09 21:26:27.055721 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.055275    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"
Sep 09 21:26:27.055721 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.055308    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx" event={"ID":"d247881e-9dfa-4a80-afb1-3b1c96acea00","Type":"ContainerDied","Data":"27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71"}
Sep 09 21:26:27.055721 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.055352    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx" event={"ID":"d247881e-9dfa-4a80-afb1-3b1c96acea00","Type":"ContainerDied","Data":"351c74261e678422b216cbeee0cc431247f4a236bf462b13f8c80827a67a7622"}
Sep 09 21:26:27.055721 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.055374    2571 scope.go:117] "RemoveContainer" containerID="27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71"
Sep 09 21:26:27.068019 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.067998    2571 scope.go:117] "RemoveContainer" containerID="27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71"
Sep 09 21:26:27.068353 ip-10-0-129-13 kubenswrapper[2571]: E0909 21:26:27.068334    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71\": container with ID starting with 27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71 not found: ID does not exist" containerID="27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71"
Sep 09 21:26:27.068403 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.068362    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71"} err="failed to get container status \"27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71\": rpc error: code = NotFound desc = could not find container \"27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71\": container with ID starting with 27bcda58271a57ab0e30a9b78e5876417dbcd02c8d211e5169dcef73da161a71 not found: ID does not exist"
Sep 09 21:26:27.077862 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.077833    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"]
Sep 09 21:26:27.089917 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.089891    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-694fed88-kserve-697d8bc5ff-2ktgx"]
Sep 09 21:26:27.257805 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:27.257780    2571 scope.go:117] "RemoveContainer" containerID="f36ee9465c078e156845a101d1108b844926a4d4dae8348fb54dfc0aedcd9628"
Sep 09 21:26:28.566508 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:26:28.566476    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d247881e-9dfa-4a80-afb1-3b1c96acea00" path="/var/lib/kubelet/pods/d247881e-9dfa-4a80-afb1-3b1c96acea00/volumes"
Sep 09 21:27:05.428864 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:05.428819    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d75fb655b-dfcc6_9925375d-085e-43f6-9067-6938b860b34b/copy-manifests/0.log"
Sep 09 21:27:05.447354 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:05.447325    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d75fb655b-dfcc6_9925375d-085e-43f6-9067-6938b860b34b/manager/0.log"
Sep 09 21:27:06.243250 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:06.243214    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29816475-zqnx8_2a4ffccb-dfe7-471c-b170-ad3cd61b5e21/cleanup/0.log"
Sep 09 21:27:06.353415 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:06.353386    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hmd4z_97276c9b-be1a-4096-8374-58b211f5692d/postgres/0.log"
Sep 09 21:27:07.456343 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:07.456310    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-8222x_d38f9345-15de-4590-b6cd-faf82fd4c4f3/kuadrant-console-plugin/0.log"
Sep 09 21:27:08.375499 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:08.375462    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcf9fffd5-7xzlb_6af00cf2-84d6-44ae-9827-2064674cd7df/kube-auth-proxy/0.log"
Sep 09 21:27:08.856377 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:08.856341    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:27:09.540231 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:09.540205    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-nf5vx_f8447f9b-fa9b-43a1-8d30-0e7618775da7/main/0.log"
Sep 09 21:27:09.648848 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:09.648815    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vssg4_b3e6df7d-f587-432e-b8ff-863913e5674e/main/0.log"
Sep 09 21:27:15.455501 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.455465    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7p779/must-gather-scqs4"]
Sep 09 21:27:15.455934 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.455891    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fe12070-0583-4b9a-a73f-56edefa17d90" containerName="main"
Sep 09 21:27:15.455934 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.455902    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fe12070-0583-4b9a-a73f-56edefa17d90" containerName="main"
Sep 09 21:27:15.455934 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.455921    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d247881e-9dfa-4a80-afb1-3b1c96acea00" containerName="main"
Sep 09 21:27:15.455934 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.455927    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d247881e-9dfa-4a80-afb1-3b1c96acea00" containerName="main"
Sep 09 21:27:15.456072 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.455993    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fe12070-0583-4b9a-a73f-56edefa17d90" containerName="main"
Sep 09 21:27:15.456072 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.456002    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d247881e-9dfa-4a80-afb1-3b1c96acea00" containerName="main"
Sep 09 21:27:15.459452 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.459427    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7p779/must-gather-scqs4"
Sep 09 21:27:15.461793 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.461771    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7p779\"/\"openshift-service-ca.crt\""
Sep 09 21:27:15.461924 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.461771    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7p779\"/\"default-dockercfg-fxqvc\""
Sep 09 21:27:15.462004 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.461917    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7p779\"/\"kube-root-ca.crt\""
Sep 09 21:27:15.474948 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.474920    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7p779/must-gather-scqs4"]
Sep 09 21:27:15.482439 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.482394    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:27:15.571507 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.571475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log" limit=1024
Sep 09 21:27:15.620668 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.620626    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl44v\" (UniqueName: \"kubernetes.io/projected/babf130d-370b-4538-a00d-1af657c9d4e4-kube-api-access-tl44v\") pod \"must-gather-scqs4\" (UID: \"babf130d-370b-4538-a00d-1af657c9d4e4\") " pod="openshift-must-gather-7p779/must-gather-scqs4"
Sep 09 21:27:15.620846 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.620764    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/babf130d-370b-4538-a00d-1af657c9d4e4-must-gather-output\") pod \"must-gather-scqs4\" (UID: \"babf130d-370b-4538-a00d-1af657c9d4e4\") " pod="openshift-must-gather-7p779/must-gather-scqs4"
Sep 09 21:27:15.721659 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.721565    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl44v\" (UniqueName: \"kubernetes.io/projected/babf130d-370b-4538-a00d-1af657c9d4e4-kube-api-access-tl44v\") pod \"must-gather-scqs4\" (UID: \"babf130d-370b-4538-a00d-1af657c9d4e4\") " pod="openshift-must-gather-7p779/must-gather-scqs4"
Sep 09 21:27:15.721817 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.721668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/babf130d-370b-4538-a00d-1af657c9d4e4-must-gather-output\") pod \"must-gather-scqs4\" (UID: \"babf130d-370b-4538-a00d-1af657c9d4e4\") " pod="openshift-must-gather-7p779/must-gather-scqs4"
Sep 09 21:27:15.722034 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.722015    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/babf130d-370b-4538-a00d-1af657c9d4e4-must-gather-output\") pod \"must-gather-scqs4\" (UID: \"babf130d-370b-4538-a00d-1af657c9d4e4\") " pod="openshift-must-gather-7p779/must-gather-scqs4"
Sep 09 21:27:15.730817 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.730777    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl44v\" (UniqueName: \"kubernetes.io/projected/babf130d-370b-4538-a00d-1af657c9d4e4-kube-api-access-tl44v\") pod \"must-gather-scqs4\" (UID: \"babf130d-370b-4538-a00d-1af657c9d4e4\") " pod="openshift-must-gather-7p779/must-gather-scqs4"
Sep 09 21:27:15.770762 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:15.770728    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7p779/must-gather-scqs4"
Sep 09 21:27:16.055403 ip-10-0-129-13 kubenswrapper[2571]: W0909 21:27:16.051300    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbabf130d_370b_4538_a00d_1af657c9d4e4.slice/crio-676739249df76c75d480e224a5572b04986cfb5706928a4f3ccecce5d56e50f3 WatchSource:0}: Error finding container 676739249df76c75d480e224a5572b04986cfb5706928a4f3ccecce5d56e50f3: Status 404 returned error can't find the container with id 676739249df76c75d480e224a5572b04986cfb5706928a4f3ccecce5d56e50f3
Sep 09 21:27:16.055403 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:16.051766    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7p779/must-gather-scqs4"]
Sep 09 21:27:16.055403 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:16.053218    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 21:27:16.341961 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:16.341873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7p779/must-gather-scqs4" event={"ID":"babf130d-370b-4538-a00d-1af657c9d4e4","Type":"ContainerStarted","Data":"676739249df76c75d480e224a5572b04986cfb5706928a4f3ccecce5d56e50f3"}
Sep 09 21:27:17.353741 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:17.353690    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7p779/must-gather-scqs4" event={"ID":"babf130d-370b-4538-a00d-1af657c9d4e4","Type":"ContainerStarted","Data":"4802667e1c3bdc5e3366869e705f65779e0c46081a4ed76e08c725fd503834f0"}
Sep 09 21:27:17.354254 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:17.353755    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7p779/must-gather-scqs4" event={"ID":"babf130d-370b-4538-a00d-1af657c9d4e4","Type":"ContainerStarted","Data":"d8f5b80e4164b92346cb394eec2340a59ccf3ed6b89597f47d6102a4630d13bc"}
Sep 09 21:27:17.373641 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:17.373577    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7p779/must-gather-scqs4" podStartSLOduration=1.6978101570000002 podStartE2EDuration="2.373553861s" totalImagesPullingTime="675.743704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:17.370546514 +0000 UTC m=+2331.341113212" watchObservedRunningTime="2026-09-09 21:27:17.373553861 +0000 UTC m=+2331.344120579"
Sep 09 21:27:18.393539 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:18.393490    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6b6sc_9efdb610-3597-487e-91c0-b0e85263d225/global-pull-secret-syncer/0.log"
Sep 09 21:27:18.495317 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:18.495266    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6lkd2_124b5fb2-9d37-4818-94f7-07af39967174/konnectivity-agent/0.log"
Sep 09 21:27:18.593598 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:18.593561    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-13.ec2.internal_bed195b806fdc3af1a773cfd2f90a22f/haproxy/0.log"
Sep 09 21:27:23.476515 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:23.476487    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-8222x_d38f9345-15de-4590-b6cd-faf82fd4c4f3/kuadrant-console-plugin/0.log"
Sep 09 21:27:25.005163 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.004625    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b491ab5c-08a4-4dd0-8073-176059571787/alertmanager/0.log"
Sep 09 21:27:25.039105 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.037927    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b491ab5c-08a4-4dd0-8073-176059571787/config-reloader/0.log"
Sep 09 21:27:25.083506 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.083476    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b491ab5c-08a4-4dd0-8073-176059571787/kube-rbac-proxy-web/0.log"
Sep 09 21:27:25.110654 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.110625    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b491ab5c-08a4-4dd0-8073-176059571787/kube-rbac-proxy/0.log"
Sep 09 21:27:25.140244 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.140214    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b491ab5c-08a4-4dd0-8073-176059571787/kube-rbac-proxy-metric/0.log"
Sep 09 21:27:25.164663 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.164630    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b491ab5c-08a4-4dd0-8073-176059571787/prom-label-proxy/0.log"
Sep 09 21:27:25.191756 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.191727    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b491ab5c-08a4-4dd0-8073-176059571787/init-config-reloader/0.log"
Sep 09 21:27:25.283009 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.282930    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-7kw6z_fe08f0b6-e8b7-4ede-85d3-489dfedde85c/kube-state-metrics/0.log"
Sep 09 21:27:25.308520 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.308497    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-7kw6z_fe08f0b6-e8b7-4ede-85d3-489dfedde85c/kube-rbac-proxy-main/0.log"
Sep 09 21:27:25.334580 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.334535    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-7kw6z_fe08f0b6-e8b7-4ede-85d3-489dfedde85c/kube-rbac-proxy-self/0.log"
Sep 09 21:27:25.372401 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.372329    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78b8c7bc6b-lf6vw_5fbc453e-193a-41d6-b01d-87365ca9b6a1/metrics-server/0.log"
Sep 09 21:27:25.410792 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.410766    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-2n8jg_77544acd-b941-4893-9ead-4b53ca7fcc99/monitoring-plugin/0.log"
Sep 09 21:27:25.560743 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.560661    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmvvb_e3fd2e5a-8d20-40c8-8d19-c958982e39c9/node-exporter/0.log"
Sep 09 21:27:25.590408 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.590362    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmvvb_e3fd2e5a-8d20-40c8-8d19-c958982e39c9/kube-rbac-proxy/0.log"
Sep 09 21:27:25.621899 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.621781    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmvvb_e3fd2e5a-8d20-40c8-8d19-c958982e39c9/init-textfile/0.log"
Sep 09 21:27:25.740583 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.740551    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-r6dlw_87c8906a-3840-4dcf-afa7-3e85753b28a0/kube-rbac-proxy-main/0.log"
Sep 09 21:27:25.766569 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.766539    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-r6dlw_87c8906a-3840-4dcf-afa7-3e85753b28a0/kube-rbac-proxy-self/0.log"
Sep 09 21:27:25.797032 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:25.796983    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-r6dlw_87c8906a-3840-4dcf-afa7-3e85753b28a0/openshift-state-metrics/0.log"
Sep 09 21:27:26.203753 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.203727    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-878c97446-lfkw9_0937c9db-c75d-49f4-81f8-f670bee27b5f/telemeter-client/0.log"
Sep 09 21:27:26.233133 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.233104    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-878c97446-lfkw9_0937c9db-c75d-49f4-81f8-f670bee27b5f/reload/0.log"
Sep 09 21:27:26.257722 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.257691    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-878c97446-lfkw9_0937c9db-c75d-49f4-81f8-f670bee27b5f/kube-rbac-proxy/0.log"
Sep 09 21:27:26.305237 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.305210    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75498d46-bxr62_97c265a5-a54d-4ed9-810b-9d81eb0fec70/thanos-query/0.log"
Sep 09 21:27:26.330282 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.330250    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75498d46-bxr62_97c265a5-a54d-4ed9-810b-9d81eb0fec70/kube-rbac-proxy-web/0.log"
Sep 09 21:27:26.360049 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.360021    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75498d46-bxr62_97c265a5-a54d-4ed9-810b-9d81eb0fec70/kube-rbac-proxy/0.log"
Sep 09 21:27:26.386488 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.386463    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75498d46-bxr62_97c265a5-a54d-4ed9-810b-9d81eb0fec70/prom-label-proxy/0.log"
Sep 09 21:27:26.408423 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.408399    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75498d46-bxr62_97c265a5-a54d-4ed9-810b-9d81eb0fec70/kube-rbac-proxy-rules/0.log"
Sep 09 21:27:26.434188 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:26.434158    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-75498d46-bxr62_97c265a5-a54d-4ed9-810b-9d81eb0fec70/kube-rbac-proxy-metrics/0.log"
Sep 09 21:27:27.288866 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:27.288832    2571 scope.go:117] "RemoveContainer" containerID="3112dee768d64ad399f9082519a8728f84d96634c8cd164228a027c8c803fd06"
Sep 09 21:27:27.305444 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:27.305418    2571 scope.go:117] "RemoveContainer" containerID="bb57adaa0859c5845c70503dacdd97cc0449ff5ff096371810442f0d3b426bf9"
Sep 09 21:27:27.816344 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:27.816296    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-mbcnl_4f8c0270-6125-42c2-a87b-f72f7483ed01/networking-console-plugin/0.log"
Sep 09 21:27:28.666046 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.666002    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"]
Sep 09 21:27:28.670225 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.670200    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.686153 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.686126    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"]
Sep 09 21:27:28.777373 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.777335    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-lib-modules\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.777555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.777407    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-sys\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.777555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.777459    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-proc\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.777555 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.777546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6ghl\" (UniqueName: \"kubernetes.io/projected/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-kube-api-access-f6ghl\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.777679 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.777576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-podres\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.878915 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.878885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-lib-modules\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.878915 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.878932    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-sys\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.879236 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.878963    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-proc\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.879236 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.879004    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6ghl\" (UniqueName: \"kubernetes.io/projected/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-kube-api-access-f6ghl\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.879236 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.879023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-podres\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.879236 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.879134    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-lib-modules\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.879236 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.879187    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-podres\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.879236 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.879224    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-proc\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.879571 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.879295    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-sys\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.888837 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.888805    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6ghl\" (UniqueName: \"kubernetes.io/projected/8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4-kube-api-access-f6ghl\") pod \"perf-node-gather-daemonset-v7cgk\" (UID: \"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4\") " pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:28.984521 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:28.984482    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:29.199844 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:29.199789    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8577558f59-5n4zk_37a8822a-c0d1-4762-a14a-5079f6caa669/console/0.log"
Sep 09 21:27:29.241150 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:29.241018    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-5n6ft_270c8650-2237-4f09-ac0e-9d7c6b803e17/download-server/0.log"
Sep 09 21:27:29.288771 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:29.287044    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"]
Sep 09 21:27:29.475899 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:29.475852    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk" event={"ID":"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4","Type":"ContainerStarted","Data":"361d22a33d54d71e566afdefa2b0954052f0dea5f436ea3e8468f6f73953e917"}
Sep 09 21:27:29.476109 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:29.475914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk" event={"ID":"8d149b06-d72e-4ccd-8b9a-59ec2a2ec3a4","Type":"ContainerStarted","Data":"dc1cde7839774d64c558af68804b1cfba1aa7e89cd13c8d1431a5f3f4d14e87a"}
Sep 09 21:27:29.476183 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:29.476116    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:29.494392 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:29.494263    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk" podStartSLOduration=1.494240337 podStartE2EDuration="1.494240337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:29.492720998 +0000 UTC m=+2343.463287716" watchObservedRunningTime="2026-09-09 21:27:29.494240337 +0000 UTC m=+2343.464807057"
Sep 09 21:27:29.972682 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:29.972640    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-zx6nb_1ae94f88-c80d-4afa-941c-e43185426d12/volume-data-source-validator/0.log"
Sep 09 21:27:31.096319 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:31.096285    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8nr68_02494caa-a335-4dd7-b8b2-571a5555e37e/dns/0.log"
Sep 09 21:27:31.131093 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:31.131045    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8nr68_02494caa-a335-4dd7-b8b2-571a5555e37e/kube-rbac-proxy/0.log"
Sep 09 21:27:31.307600 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:31.307572    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nlzcr_76649ca0-30fa-40ea-8854-2ddab01b1692/dns-node-resolver/0.log"
Sep 09 21:27:31.961567 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:31.961533    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rp6l9_237b6000-010e-4262-9ed4-9cfd02d1df7d/node-ca/0.log"
Sep 09 21:27:33.183572 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:33.183544    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcf9fffd5-7xzlb_6af00cf2-84d6-44ae-9827-2064674cd7df/kube-auth-proxy/0.log"
Sep 09 21:27:33.352039 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:33.352006    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:27:34.269519 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:34.269464    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nrrnq_d1c6a852-7538-44c4-aa73-0b3faa8a9049/serve-healthcheck-canary/0.log"
Sep 09 21:27:34.796878 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:34.796844    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-jlprg_caaa9f3d-3587-4330-ac7f-e3982c2c09a4/insights-operator/0.log"
Sep 09 21:27:34.797137 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:34.797004    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-jlprg_caaa9f3d-3587-4330-ac7f-e3982c2c09a4/insights-operator/1.log"
Sep 09 21:27:34.822228 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:34.822155    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-58h2w_0e389e63-e95d-4cac-bef6-7bd051dfd17d/kube-rbac-proxy/0.log"
Sep 09 21:27:34.847133 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:34.847092    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-58h2w_0e389e63-e95d-4cac-bef6-7bd051dfd17d/exporter/0.log"
Sep 09 21:27:34.869391 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:34.869361    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-58h2w_0e389e63-e95d-4cac-bef6-7bd051dfd17d/extractor/0.log"
Sep 09 21:27:35.494938 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:35.494908    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7p779/perf-node-gather-daemonset-v7cgk"
Sep 09 21:27:37.969801 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:37.969765    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d75fb655b-dfcc6_9925375d-085e-43f6-9067-6938b860b34b/manager/0.log"
Sep 09 21:27:37.991162 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:37.991104    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d75fb655b-dfcc6_9925375d-085e-43f6-9067-6938b860b34b/copy-manifests/0.log"
Sep 09 21:27:39.436651 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:39.436551    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-544fcc9ff7-4tnf7_e3e51597-2888-44c1-882e-1dfbe4beb0da/manager/0.log"
Sep 09 21:27:39.559059 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:39.559030    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcf9fffd5-7xzlb_6af00cf2-84d6-44ae-9827-2064674cd7df/kube-auth-proxy/0.log"
Sep 09 21:27:39.698703 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:39.698671    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:27:41.438586 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:41.438490    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bcf9fffd5-7xzlb_6af00cf2-84d6-44ae-9827-2064674cd7df/kube-auth-proxy/0.log"
Sep 09 21:27:41.581397 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:41.581363    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-g7qxr_03afa2b6-eccc-4aa4-9979-b3b725de7207/payload-processing/0.log"
Sep 09 21:27:45.296116 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:45.296069    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-dkb8z_54399796-4e0e-4fb9-803a-7d5c20f66b4d/kube-storage-version-migrator-operator/1.log"
Sep 09 21:27:45.297472 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:45.297452    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-dkb8z_54399796-4e0e-4fb9-803a-7d5c20f66b4d/kube-storage-version-migrator-operator/0.log"
Sep 09 21:27:46.462007 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:46.461975    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j67z_3f86e245-9338-4c00-ae1a-129411488355/kube-multus-additional-cni-plugins/0.log"
Sep 09 21:27:46.483457 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:46.483431    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j67z_3f86e245-9338-4c00-ae1a-129411488355/egress-router-binary-copy/0.log"
Sep 09 21:27:46.506374 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:46.506347    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j67z_3f86e245-9338-4c00-ae1a-129411488355/cni-plugins/0.log"
Sep 09 21:27:46.529388 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:46.529350    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j67z_3f86e245-9338-4c00-ae1a-129411488355/bond-cni-plugin/0.log"
Sep 09 21:27:46.551356 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:46.551321    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j67z_3f86e245-9338-4c00-ae1a-129411488355/routeoverride-cni/0.log"
Sep 09 21:27:46.575261 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:46.575231    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j67z_3f86e245-9338-4c00-ae1a-129411488355/whereabouts-cni-bincopy/0.log"
Sep 09 21:27:46.596374 ip-10-0-129-13 kubenswrapper[2571]: I0909 21:27:46.596347    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7j67z_3f86e245-9338-4c00-ae1a-129411488355/whereabouts-cni/0.log"