Sep 16 16:34:34.471106 ip-10-0-139-51 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 16:34:34.471116 ip-10-0-139-51 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 16:34:34.471123 ip-10-0-139-51 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 16:34:34.471279 ip-10-0-139-51 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 16:34:44.720748 ip-10-0-139-51 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 16:34:44.720766 ip-10-0-139-51 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5b2a3c0ac52b41beafd7d8f6ec754819 --
Sep 16 16:36:24.391901 ip-10-0-139-51 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 16:36:24.743564 ip-10-0-139-51 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 16:36:24.743564 ip-10-0-139-51 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 16:36:24.743564 ip-10-0-139-51 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 16:36:24.743564 ip-10-0-139-51 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 16:36:24.743564 ip-10-0-139-51 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 16:36:24.745199 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.745111    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 16:36:24.750226 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750206    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:24.750226 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750222    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:24.750226 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750228    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:24.750226 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750232    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750237    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750241    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750245    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750249    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750253    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750258    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750262    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750266    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750270    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750278    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750282    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750286    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750290    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750294    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750298    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750303    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750307    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750311    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750316    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:24.750523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750320    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750324    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750328    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750332    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750338    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750343    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750348    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750353    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750358    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750362    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750366    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750370    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750374    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750380    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750385    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750389    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750393    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750397    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750401    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750406    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:24.751368 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750410    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750414    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750419    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750423    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750427    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750432    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750436    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750440    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750444    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750448    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750452    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750456    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750460    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750464    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750468    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750474    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750479    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750484    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750488    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750495    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:24.752225 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750500    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750505    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750510    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750516    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750520    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750541    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750547    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750553    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750558    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750563    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750567    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750571    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750575    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750579    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750584    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750588    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750595    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750601    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750605    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:24.753074 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750609    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750614    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750618    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750623    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750627    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750631    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750635    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750639    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750643    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750647    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750654    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750660    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750665    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750669    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750673    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750677    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.750682    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751891    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751901    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751908    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:24.753906 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751913    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751919    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751923    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751927    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751932    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751935    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751939    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751944    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751948    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751952    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751956    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751960    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751965    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751969    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751973    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751977    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751982    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751986    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751990    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751994    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:24.754659 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.751998    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752003    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752007    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752011    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752015    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752019    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752024    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752028    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752033    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752037    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752041    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752046    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752050    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752054    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752060    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752064    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752068    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752072    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752076    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:24.755281 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752080    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752084    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752088    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752093    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752096    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752101    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752105    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752109    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752114    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752118    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752122    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752126    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752130    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752134    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752138    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752142    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752147    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752151    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752155    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752161    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:24.755797 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752165    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752169    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752173    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752177    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752183    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752187    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752193    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752199    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752204    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752209    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752216    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752222    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752227    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752231    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752235    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752240    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752244    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752248    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752252    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:24.756301 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752257    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752264    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752269    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752274    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752279    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752283    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752288    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752293    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752299    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752304    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752308    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752313    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752318    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752323    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752328    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752333    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752338    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752343    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752954    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:24.756800 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752962    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752967    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752973    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752977    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752982    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752986    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752990    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752994    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.752998    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753003    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753007    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753011    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753015    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753020    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753024    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753028    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753032    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753037    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753042    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:24.757270 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753046    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753051    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753055    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753059    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753065    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753070    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753076    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753080    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753085    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753089    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753093    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753097    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753101    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753106    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753110    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753116    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753121    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753126    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753130    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753134    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:24.757757 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753138    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753142    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753147    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753151    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753155    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753159    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753164    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753168    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753173    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753177    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753181    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753185    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753189    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753193    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753198    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753202    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753206    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753210    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753214    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753218    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:24.758252 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753222    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753228    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753233    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753237    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753242    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753246    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753250    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753254    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753260    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753266    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753270    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753275    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753279    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753283    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753288    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753292    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753297    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753301    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753306    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753310    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:24.758759 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753314    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753318    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753322    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753327    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753331    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753339    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753343    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753347    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753351    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753356    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753360    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753365    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753369    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753373    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753377    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753381    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753385    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753390    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.753393    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753498    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 16:36:24.759280 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753567    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753578    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753586    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753593    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753598    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753604    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753610    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753616    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753621    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753627    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753632    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753637    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753642    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753647    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753652    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753657    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753662    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753670    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753675    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753680    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753685    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753690    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753697    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 16:36:24.759792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753702    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753707    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753712    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753717    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753722    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753726    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753731    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753736    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753742    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753747    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753752    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753756    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753762    2577 flags.go:64] FLAG: --enable-server="true"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753767    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753774    2577 flags.go:64] FLAG: --event-burst="100"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753779    2577 flags.go:64] FLAG: --event-qps="50"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753784    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753790    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753795    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753801    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753806    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753810    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753815    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753820    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753824    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 16:36:24.760391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753829    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753836    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753840    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753845    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753850    2577 flags.go:64] FLAG: --feature-gates=""
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753856    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753861    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753865    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753871    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753876    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753881    2577 flags.go:64] FLAG: --help="false"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753885    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-139-51.ec2.internal"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753890    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753895    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753900    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753905    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753910    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753915    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753919    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753923    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753928    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753933    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753939    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753943    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 16 16:36:24.761039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753948    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753952    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753958    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753962    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753967    2577 flags.go:64] FLAG: --lock-file=""
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753972    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753976    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753981    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.753996    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754002    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754006    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754011    2577 flags.go:64] FLAG: --logging-format="text"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754015    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754021    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754026    2577 flags.go:64] FLAG: --manifest-url=""
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754030    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754037    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754041    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754048    2577 flags.go:64] FLAG: --max-pods="110"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754052    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754057    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754063    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754068    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754073    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754078    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 16:36:24.761677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754082    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754093    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754098    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754103    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754109    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754113    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754122    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754127    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754132    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754139    2577 flags.go:64] FLAG: --port="10250"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754144    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754149    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07ddda7f516e06c58"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754154    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754160    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754164    2577 flags.go:64] FLAG: --register-node="true"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754169    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754175    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754179    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754184    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754189    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754195    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754200    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754205    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754210    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754215    2577 flags.go:64] FLAG: --runonce="false"
Sep 16 16:36:24.762453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754219    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754225    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754230    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754235    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754239    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754244    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754249    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754254    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754258    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754262    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754266    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754271    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754277    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754282    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754287    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754295    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754301    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754306    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754312    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754316    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754321    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754326    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754331    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754336    2577 flags.go:64] FLAG: --v="2"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754342    2577 flags.go:64] FLAG: --version="false"
Sep 16 16:36:24.763363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754349    2577 flags.go:64] FLAG: --vmodule=""
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754354    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.754360    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754554    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754569    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754573    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754577    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754581    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754584    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754587    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754590    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754593    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754596    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754599    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754602    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754605    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754607    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754610    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754613    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754616    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:24.764004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754619    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754621    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754624    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754627    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754629    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754634    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754638    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754641    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754644    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754647    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754650    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754653    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754656    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754659    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754662    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754665    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754672    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754674    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754677    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754680    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:24.764558 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754683    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754686    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754688    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754691    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754694    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754696    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754699    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754702    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754704    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754707    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754710    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754712    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754715    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754717    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754720    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754722    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754725    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754728    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754731    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754734    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:24.765052 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754736    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754739    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754741    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754744    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754747    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754749    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754752    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754755    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754759    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754762    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754765    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754767    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754771    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754775    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754778    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754782    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754786    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754789    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754791    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:24.765622 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754794    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754797    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754800    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754803    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754806    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754808    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754811    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754814    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754817    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754819    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754822    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754824    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754827    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754830    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754832    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754835    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754837    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754840    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754842    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754845    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:24.766093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754848    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754852    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.754855    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.755465    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.764116    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.764239    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.764911    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.764919    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.764975    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.764981    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.764984    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.764988    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.764991    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.764994    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.764996    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:24.766597 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.764999    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765002    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765004    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765007    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765010    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765013    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765016    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765019    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765021    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765024    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765026    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765029    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765031    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765034    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765036    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765039    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765042    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765046    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765049    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:24.766965 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765052    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765056    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765061    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765064    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765068    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765072    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765074    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765077    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765080    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765082    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765086    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765090    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765093    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765096    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765099    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765102    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765105    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765108    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765111    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:24.767428 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765114    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765116    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765119    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765122    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765125    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765128    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765131    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765134    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765136    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765139    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765142    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765145    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765147    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765150    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765153    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765157    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765159    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765162    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765164    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765167    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:24.767944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765170    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765172    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765175    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765177    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765180    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765182    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765184    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765187    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765189    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765192    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765195    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765197    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765200    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765203    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765206    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765209    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765212    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765214    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765217    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765219    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765222    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:24.768427 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765224    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765227    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765230    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765232    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765235    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765237    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765240    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765243    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765246    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765248    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765251    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765254    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765257    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.765261    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765395    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 16:36:24.768944 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765401    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765404    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765407    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765410    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765414    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765417    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765419    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765422    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765424    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765427    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765430    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765433    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765435    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765438    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765442    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765446    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765449    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765451    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765455    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765458    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 16:36:24.769313 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765460    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765463    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765466    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765469    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765471    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765474    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765477    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765481    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765484    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765487    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765490    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765493    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765496    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765499    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765501    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765504    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765507    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765510    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765514    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 16:36:24.769916 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765517    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765520    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765522    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765539    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765542    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765546    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765548    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765551    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765554    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765556    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765559    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765561    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765564    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765567    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765569    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765572    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765575    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765577    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765580    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765582    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 16:36:24.770369 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765585    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765587    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765590    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765593    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765595    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765598    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765600    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765602    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765605    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765607    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765610    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765612    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765615    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765617    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765620    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765623    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765626    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765628    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765632    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765635    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 16:36:24.770867 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765638    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765641    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765644    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765647    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765656    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765659    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765661    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765664    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765667    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765669    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765672    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765675    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765677    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765680    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765682    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765685    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765688    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765690    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 16:36:24.771346 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:24.765693    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 16:36:24.771803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.765697    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 16:36:24.771803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.765816    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 16:36:24.771803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.770305    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 16:36:24.771803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.771138    2577 server.go:1019] "Starting client certificate rotation"
Sep 16 16:36:24.771803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.771237    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 16:36:24.771803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.771276    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 16:36:24.773815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.773801    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 16:36:24.776510 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.776492    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 16:36:24.787174 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.787151    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 16 16:36:24.790847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.790833    2577 log.go:25] "Validated CRI v1 image API"
Sep 16 16:36:24.792314 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.792299    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 16:36:24.795095 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.795070    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a3aff196-0ff8-4b05-a563-3afbc19a9fcc:/dev/nvme0n1p4 de3b50ce-e16a-4b59-b582-fa9e03a98f46:/dev/nvme0n1p3]
Sep 16 16:36:24.795153 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.795097    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 16:36:24.801389 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.801288    2577 manager.go:216] Machine: {Timestamp:2026-09-16 16:36:24.799942175 +0000 UTC m=+0.296815290 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3085058 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f3c5a98476f1756ea7847b6980f14 SystemUUID:ec2f3c5a-9847-6f17-56ea-7847b6980f14 BootID:5b2a3c0a-c52b-41be-afd7-d8f6ec754819 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ca:a3:d7:67 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ca:a3:d7:67 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:c5:64:9b:59:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 16:36:24.801389 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.801385    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 16:36:24.801498 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.801487    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 16:36:24.804573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.804546    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 16:36:24.804701 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.804575    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-51.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 16:36:24.805948 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.805933    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 16:36:24.805948 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.805946    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 16:36:24.806008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.805954    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 16:36:24.806111 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.806092    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 16:36:24.806738 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.806725    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 16:36:24.808073 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.808046    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 16:36:24.812327 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812312    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 16:36:24.812447 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812437    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 16:36:24.812479 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812467    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 16:36:24.812479 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812477    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 16:36:24.812556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812494    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 16:36:24.812556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812508    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 16 16:36:24.812556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812519    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 16:36:24.812876 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812864    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 16:36:24.812910 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.812888    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 16:36:24.814872 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.814855    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 16:36:24.817089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.817073    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 16:36:24.817165 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.817113    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 16:36:24.819337 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819322    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819344    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819353    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819361    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819374    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819382    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819401    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819409    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819419    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 16:36:24.819425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819427    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 16:36:24.819720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819439    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 16:36:24.819720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.819447    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 16:36:24.820177 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.820166    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 16:36:24.820231 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.820179    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 16:36:24.824325 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.824309    2577 server.go:1268] "Started kubelet"
Sep 16 16:36:24.824458 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.824402    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 16:36:24.824563 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.824547    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 16:36:24.824806 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.824786    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 16:36:24.824946 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.824927    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-51.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 16:36:24.825125 ip-10-0-139-51 systemd[1]: Started Kubernetes Kubelet.
Sep 16 16:36:24.825488 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.825461    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-51.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 16:36:24.825488 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.825457    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 16:36:24.826395 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.826380    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 16:36:24.827048 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.827007    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 16:36:24.827179 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.827161    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cxr8t"
Sep 16 16:36:24.829984 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.829177    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-51.ec2.internal.18d5da6c1ccee7c8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-51.ec2.internal,UID:ip-10-0-139-51.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-51.ec2.internal,},FirstTimestamp:2026-09-16 16:36:24.824285128 +0000 UTC m=+0.321158243,LastTimestamp:2026-09-16 16:36:24.824285128 +0000 UTC m=+0.321158243,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-51.ec2.internal,}"
Sep 16 16:36:24.831453 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.831408    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 16:36:24.832064 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.832038    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 16:36:24.832855 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.832839    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 16:36:24.832937 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.832923    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:24.833384 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833367    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 16:36:24.833384 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833387    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 16:36:24.833547 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833409    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 16:36:24.833547 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833486    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 16:36:24.833547 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833494    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 16:36:24.833685 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833664    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 16:36:24.833685 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833673    2577 factory.go:55] Registering systemd factory
Sep 16 16:36:24.833685 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833679    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 16 16:36:24.833883 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833866    2577 factory.go:153] Registering CRI-O factory
Sep 16 16:36:24.833883 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833883    2577 factory.go:223] Registration of the crio container factory successfully
Sep 16 16:36:24.834000 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833897    2577 factory.go:103] Registering Raw factory
Sep 16 16:36:24.834000 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.833919    2577 manager.go:1196] Started watching for new ooms in manager
Sep 16 16:36:24.834625 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.834609    2577 manager.go:349] Starting recovery of all containers
Sep 16 16:36:24.834915 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.834884    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-51.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 16:36:24.835070 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.835048    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 16:36:24.835185 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.835165    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cxr8t"
Sep 16 16:36:24.845142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.845032    2577 manager.go:354] Recovery completed
Sep 16 16:36:24.849072 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.849058    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:24.850939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.850924    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:24.851031 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.850951    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:24.851031 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.850964    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:24.851514 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.851499    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 16:36:24.851615 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.851506    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:24.851615 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.851524    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 16:36:24.851615 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.851589    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 16:36:24.852181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.852163    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:24.852290 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.852190    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:24.852290 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.852199    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:24.853605 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.853592    2577 policy_none.go:49] "None policy: Start"
Sep 16 16:36:24.853665 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.853610    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 16:36:24.853665 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.853622    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 16:36:24.856673 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.855627    2577 policy_none.go:47] "Start"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.885118    2577 manager.go:337] "Starting Device Plugin manager"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.885148    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.885161    2577 server.go:87] "Starting device plugin registration server"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.885382    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.885396    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.885467    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.885563    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.885573    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.886003    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 16:36:24.911947 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.886049    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:24.969392 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.969370    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 16:36:24.970447 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.970429    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 16:36:24.970554 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.970478    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 16:36:24.970554 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.970500    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 16:36:24.970647 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.970552    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 16:36:24.972760 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.972742    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:24.985629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.985615    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:24.986708 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.986691    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:24.986798 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.986719    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:24.986798 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.986729    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:24.986798 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.986750    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-51.ec2.internal"
Sep 16 16:36:24.995660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:24.995601    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-51.ec2.internal"
Sep 16 16:36:24.995660 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:24.995626    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-51.ec2.internal\": node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.047112 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.047090    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.070886 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.070863    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal"]
Sep 16 16:36:25.070940 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.070928    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:25.071746 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.071732    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:25.071798 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.071760    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:25.071798 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.071773    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:25.073904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.073892    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:25.074024 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.074011    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.074059 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.074041    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:25.074973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.074955    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:25.074973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.074963    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:25.075118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.074981    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:25.075118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.074990    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:25.075118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.075009    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:25.075118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.074992    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:25.077117 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.077104    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.077168 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.077126    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 16:36:25.077787 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.077770    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 16:36:25.077848 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.077805    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 16:36:25.077848 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.077818    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeHasSufficientPID"
Sep 16 16:36:25.100097 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.100072    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-51.ec2.internal\" not found" node="ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.104638 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.104624    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-51.ec2.internal\" not found" node="ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.134830 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.134805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae7a50717792db9bcd51d851252d6e22-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal\" (UID: \"ae7a50717792db9bcd51d851252d6e22\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.134936 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.134843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae7a50717792db9bcd51d851252d6e22-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal\" (UID: \"ae7a50717792db9bcd51d851252d6e22\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.134936 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.134864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/29c4f9a3e72c82af2027e396c27340f6-config\") pod \"kube-apiserver-proxy-ip-10-0-139-51.ec2.internal\" (UID: \"29c4f9a3e72c82af2027e396c27340f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.148183 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.148149    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.235887 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.235858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae7a50717792db9bcd51d851252d6e22-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal\" (UID: \"ae7a50717792db9bcd51d851252d6e22\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.235979 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.235903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae7a50717792db9bcd51d851252d6e22-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal\" (UID: \"ae7a50717792db9bcd51d851252d6e22\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.235979 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.235912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ae7a50717792db9bcd51d851252d6e22-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal\" (UID: \"ae7a50717792db9bcd51d851252d6e22\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.235979 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.235928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/29c4f9a3e72c82af2027e396c27340f6-config\") pod \"kube-apiserver-proxy-ip-10-0-139-51.ec2.internal\" (UID: \"29c4f9a3e72c82af2027e396c27340f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.235979 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.235963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/29c4f9a3e72c82af2027e396c27340f6-config\") pod \"kube-apiserver-proxy-ip-10-0-139-51.ec2.internal\" (UID: \"29c4f9a3e72c82af2027e396c27340f6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.236115 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.235991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae7a50717792db9bcd51d851252d6e22-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal\" (UID: \"ae7a50717792db9bcd51d851252d6e22\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.248970 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.248918    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.349713 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.349670    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.402847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.402822    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.409119 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.409101    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.450393 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.450357    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.550838 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.550770    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.651749 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.651721    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.751913 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:25.751886    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-51.ec2.internal\" not found"
Sep 16 16:36:25.771525 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.771486    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 16:36:25.771714 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.771653    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 16:36:25.835200 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.835139    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:25.841298 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.841270    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 16:31:25 +0000 UTC" deadline="2028-03-07 16:59:43.057723385 +0000 UTC"
Sep 16 16:36:25.841298 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.841297    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12912h23m17.216429517s"
Sep 16 16:36:25.934131 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.934100    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.949892 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.949877    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 16:36:25.951524 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.951511    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal"
Sep 16 16:36:25.960999 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:25.960981    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 16:36:26.091898 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.091680    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:26.117435 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:26.117399    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c4f9a3e72c82af2027e396c27340f6.slice/crio-daea4980cbad4a905968cd5d6329057ca7b9df95cfa4f2b25fc9b2c30c18d6df WatchSource:0}: Error finding container daea4980cbad4a905968cd5d6329057ca7b9df95cfa4f2b25fc9b2c30c18d6df: Status 404 returned error can't find the container with id daea4980cbad4a905968cd5d6329057ca7b9df95cfa4f2b25fc9b2c30c18d6df
Sep 16 16:36:26.118130 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:26.118103    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7a50717792db9bcd51d851252d6e22.slice/crio-32fcf63297a5647897cb731c45124652b047107019ed3c246343fdca9c1c6baa WatchSource:0}: Error finding container 32fcf63297a5647897cb731c45124652b047107019ed3c246343fdca9c1c6baa: Status 404 returned error can't find the container with id 32fcf63297a5647897cb731c45124652b047107019ed3c246343fdca9c1c6baa
Sep 16 16:36:26.124641 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.124627    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:36:26.282388 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.282361    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:26.813539 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.813507    2577 apiserver.go:52] "Watching apiserver"
Sep 16 16:36:26.822388 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.822364    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 16:36:26.822784 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.822752    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-htltb","openshift-image-registry/node-ca-4ft9d","openshift-multus/multus-additional-cni-plugins-p7zhj","openshift-multus/multus-fpwb6","openshift-multus/network-metrics-daemon-9wkss","kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal","openshift-network-diagnostics/network-check-target-rzt9z","openshift-network-operator/iptables-alerter-wqq6j","openshift-ovn-kubernetes/ovnkube-node-tbzhg","kube-system/global-pull-secret-syncer-b2pqs","kube-system/konnectivity-agent-g4mhj","openshift-cluster-node-tuning-operator/tuned-h9l5b"]
Sep 16 16:36:26.825574 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.825550    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.828091 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.828066    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:26.830748 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.830728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.831507 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.831365    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 16:36:26.831600 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.831527    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 16:36:26.831600 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.831551    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2xcn8\""
Sep 16 16:36:26.831949 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.831927    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 16:36:26.833418 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.832925    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 16:36:26.833418 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.833406    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.834962 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.834937    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 16:36:26.835832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.835812    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 16:36:26.835906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.835816    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 16:36:26.835906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.835839    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 16:36:26.835973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.835932    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 16:36:26.835973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.835948    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 16:36:26.836087 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.836062    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pbcsj\""
Sep 16 16:36:26.836183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.836163    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 16:36:26.836183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.836174    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hj6r7\""
Sep 16 16:36:26.836395 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.836374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:26.836496 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:26.836466    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:26.838230 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.838209    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 16:36:26.838503 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.838481    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m255f\""
Sep 16 16:36:26.840503 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.840481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.842748 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.842724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:26.842841 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:26.842800    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:26.844792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.844772    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 16:36:26.844792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.844786    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h4mxg\""
Sep 16 16:36:26.845140 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.845034    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:36:26.847204 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847183    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.847306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-registration-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.847306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cni-binary-copy\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.847306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-cni-bin\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.847306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-kubelet\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.847306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847301    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvb8j\" (UniqueName: \"kubernetes.io/projected/6ebfc096-7519-4787-a95a-b39896f1123d-kube-api-access-lvb8j\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cnibin\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-cni-multus\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847397    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-system-cni-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e30f3f32-2dd2-4c5c-a9a9-03328722a78f-agent-certs\") pod \"konnectivity-agent-g4mhj\" (UID: \"e30f3f32-2dd2-4c5c-a9a9-03328722a78f\") " pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847438    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-system-cni-dir\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-conf-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.847584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-multus-certs\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847767    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-cnibin\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-socket-dir-parent\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e30f3f32-2dd2-4c5c-a9a9-03328722a78f-konnectivity-ca\") pod \"konnectivity-agent-g4mhj\" (UID: \"e30f3f32-2dd2-4c5c-a9a9-03328722a78f\") " pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-k8s-cni-cncf-io\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ebfc096-7519-4787-a95a-b39896f1123d-multus-daemon-config\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847905    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-cni-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-netns\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-socket-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-device-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.847982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xps5r\" (UniqueName: \"kubernetes.io/projected/b14f8e9c-19ed-4cda-b9f2-91142a85624a-kube-api-access-xps5r\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncgsh\" (UniqueName: \"kubernetes.io/projected/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-kube-api-access-ncgsh\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848051    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-sys-fs\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.848098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.848849 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.848849 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ebfc096-7519-4787-a95a-b39896f1123d-cni-binary-copy\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848849 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-hostroot\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848849 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-etc-kubernetes\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.848849 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.848425    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcsmb\" (UniqueName: \"kubernetes.io/projected/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-kube-api-access-wcsmb\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:26.849247 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.849220    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:26.851478 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.851203    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 16:36:26.851478 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.851268    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m9d2z\""
Sep 16 16:36:26.851887 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.851864    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 16:36:26.853723 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.853592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:26.855462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.855138    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c6bp9\""
Sep 16 16:36:26.855462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.855214    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 16:36:26.856983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.855946    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 16:36:26.856983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.856228    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 16:36:26.857147 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.857084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.860219 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.860197    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 16:36:26.860340 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.860274    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 16:36:26.860565 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.860529    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 16:36:26.860652 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.860580    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:26.860719 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:26.860643    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:26.860809 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.860791    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-996wx\""
Sep 16 16:36:26.860873 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.860819    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:36:26.861658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.861484    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 16:36:26.861658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.861516    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 16:36:26.862664 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.862642    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-srf27\""
Sep 16 16:36:26.862774 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.862751    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 16:36:26.862906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.862874    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 16:36:26.863046 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.862646    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 16:36:26.872684 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.872661    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 16:36:26.926128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.926090    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7f2s9"
Sep 16 16:36:26.934780 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.934757    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 16:36:26.939389 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.939368    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7f2s9"
Sep 16 16:36:26.949065 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-log-socket\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.949150 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-ovnkube-config\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.949150 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949098    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e30f3f32-2dd2-4c5c-a9a9-03328722a78f-agent-certs\") pod \"konnectivity-agent-g4mhj\" (UID: \"e30f3f32-2dd2-4c5c-a9a9-03328722a78f\") " pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:26.949150 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-multus-certs\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.949150 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83be222b-05e2-44b0-90c5-bf387e175d3e-tmp-dir\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:26.949285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-tuned\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.949285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.949285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.949285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-socket-dir-parent\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.949285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83be222b-05e2-44b0-90c5-bf387e175d3e-hosts-file\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:26.949285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-host\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.949285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949275    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-run-ovn-kubernetes\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-ovnkube-script-lib\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysctl-d\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e30f3f32-2dd2-4c5c-a9a9-03328722a78f-konnectivity-ca\") pod \"konnectivity-agent-g4mhj\" (UID: \"e30f3f32-2dd2-4c5c-a9a9-03328722a78f\") " pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-k8s-cni-cncf-io\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ebfc096-7519-4787-a95a-b39896f1123d-multus-daemon-config\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-host\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-host-slash\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-modprobe-d\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-cni-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.949567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-socket-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-iptables-alerter-script\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-slash\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-node-log\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949871    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ebfc096-7519-4787-a95a-b39896f1123d-cni-binary-copy\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949896    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-hostroot\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdq6m\" (UniqueName: \"kubernetes.io/projected/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-kube-api-access-cdq6m\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-systemd-units\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.949973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-etc-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.950016 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcsmb\" (UniqueName: \"kubernetes.io/projected/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-kube-api-access-wcsmb\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:26.950526 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950025    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.950526 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-registration-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.950526 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cni-binary-copy\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.950526 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvb8j\" (UniqueName: \"kubernetes.io/projected/6ebfc096-7519-4787-a95a-b39896f1123d-kube-api-access-lvb8j\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.950526 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-var-lib-kubelet\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.950526 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/09bce51b-fc68-45c5-8a20-58f1361b3790-ovn-node-metrics-cert\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.950526 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:26.950845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-socket-dir-parent\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.950845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.950845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-cni-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.950845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950766    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-hostroot\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.950845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950813    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-registration-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.951001 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-k8s-cni-cncf-io\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951001 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-socket-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.951001 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.950903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-multus-certs\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951104 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:26.951045    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:26.951104 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ebfc096-7519-4787-a95a-b39896f1123d-cni-binary-copy\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951170 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ebfc096-7519-4787-a95a-b39896f1123d-multus-daemon-config\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951205 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cnibin\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.951239 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-cni-multus\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951239 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:26.951233    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:27.451216792 +0000 UTC m=+2.948089895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:26.951311 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-kubernetes\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.951311 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-cni-multus\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951382 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951289    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 16:36:26.951382 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kq7z\" (UniqueName: \"kubernetes.io/projected/09bce51b-fc68-45c5-8a20-58f1361b3790-kube-api-access-9kq7z\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.951382 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951349    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:26.951382 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e30f3f32-2dd2-4c5c-a9a9-03328722a78f-konnectivity-ca\") pod \"konnectivity-agent-g4mhj\" (UID: \"e30f3f32-2dd2-4c5c-a9a9-03328722a78f\") " pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:26.951382 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-run\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.951566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951410    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cnibin\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.951566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951468    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-system-cni-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-system-cni-dir\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.951740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-system-cni-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-conf-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-system-cni-dir\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.951740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cni-binary-copy\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.951740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-multus-conf-dir\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.951740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-run-netns\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.951740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951723    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-cni-bin\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951778    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-cnibin\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951837    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-serviceca\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-cnibin\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-cni-netd\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.951978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysconfig\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-sys\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-ovn\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.952086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-env-overrides\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysctl-conf\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nnx5\" (UniqueName: \"kubernetes.io/projected/83be222b-05e2-44b0-90c5-bf387e175d3e-kube-api-access-8nnx5\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c29mj\" (UniqueName: \"kubernetes.io/projected/6838d84d-a423-44bc-92a2-a57b635f6a51-kube-api-access-c29mj\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-kubelet\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d52fd17-9aa5-4d34-a16e-16220c51a87c-kubelet-config\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-netns\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-device-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xps5r\" (UniqueName: \"kubernetes.io/projected/b14f8e9c-19ed-4cda-b9f2-91142a85624a-kube-api-access-xps5r\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgsh\" (UniqueName: \"kubernetes.io/projected/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-kube-api-access-ncgsh\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8svxn\" (UniqueName: \"kubernetes.io/projected/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-kube-api-access-8svxn\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952357    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-var-lib-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-run-netns\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d52fd17-9aa5-4d34-a16e-16220c51a87c-dbus\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-sys-fs\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.952599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-etc-kubernetes\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952531    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-lib-modules\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-systemd\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-device-dir\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952597    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-cni-bin\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b14f8e9c-19ed-4cda-b9f2-91142a85624a-sys-fs\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-cni-bin\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-kubelet\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-etc-kubernetes\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952700    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6838d84d-a423-44bc-92a2-a57b635f6a51-tmp\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-systemd\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952699    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebfc096-7519-4787-a95a-b39896f1123d-host-var-lib-kubelet\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.952979    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.953434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.953125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.956177 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.956156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e30f3f32-2dd2-4c5c-a9a9-03328722a78f-agent-certs\") pod \"konnectivity-agent-g4mhj\" (UID: \"e30f3f32-2dd2-4c5c-a9a9-03328722a78f\") " pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:26.968169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.968145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncgsh\" (UniqueName: \"kubernetes.io/projected/d2570fd8-d0e6-4ce4-a62c-65e9b5c22533-kube-api-access-ncgsh\") pod \"multus-additional-cni-plugins-p7zhj\" (UID: \"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533\") " pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:26.970474 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.970449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcsmb\" (UniqueName: \"kubernetes.io/projected/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-kube-api-access-wcsmb\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:26.972033 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.971987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvb8j\" (UniqueName: \"kubernetes.io/projected/6ebfc096-7519-4787-a95a-b39896f1123d-kube-api-access-lvb8j\") pod \"multus-fpwb6\" (UID: \"6ebfc096-7519-4787-a95a-b39896f1123d\") " pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:26.972586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.972531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xps5r\" (UniqueName: \"kubernetes.io/projected/b14f8e9c-19ed-4cda-b9f2-91142a85624a-kube-api-access-xps5r\") pod \"aws-ebs-csi-driver-node-fcfrh\" (UID: \"b14f8e9c-19ed-4cda-b9f2-91142a85624a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:26.979506 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.979461    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal" event={"ID":"29c4f9a3e72c82af2027e396c27340f6","Type":"ContainerStarted","Data":"daea4980cbad4a905968cd5d6329057ca7b9df95cfa4f2b25fc9b2c30c18d6df"}
Sep 16 16:36:26.980900 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:26.980834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal" event={"ID":"ae7a50717792db9bcd51d851252d6e22","Type":"ContainerStarted","Data":"32fcf63297a5647897cb731c45124652b047107019ed3c246343fdca9c1c6baa"}
Sep 16 16:36:27.055343 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83be222b-05e2-44b0-90c5-bf387e175d3e-tmp-dir\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:27.055533 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-tuned\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.055533 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.055533 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.055533 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83be222b-05e2-44b0-90c5-bf387e175d3e-hosts-file\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:27.055533 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-host\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.055828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055595    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83be222b-05e2-44b0-90c5-bf387e175d3e-hosts-file\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:27.055828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-host\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.055828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.055828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-run-ovn-kubernetes\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.055828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055776    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83be222b-05e2-44b0-90c5-bf387e175d3e-tmp-dir\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:27.055828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-run-ovn-kubernetes\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-ovnkube-script-lib\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysctl-d\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-host\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-host-slash\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-modprobe-d\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysctl-d\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056099    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-iptables-alerter-script\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:27.056166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-host-slash\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-host\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-slash\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-node-log\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-slash\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdq6m\" (UniqueName: \"kubernetes.io/projected/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-kube-api-access-cdq6m\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-systemd-units\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-etc-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-modprobe-d\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-etc-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-node-log\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-var-lib-kubelet\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/09bce51b-fc68-45c5-8a20-58f1361b3790-ovn-node-metrics-cert\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-ovnkube-script-lib\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-systemd-units\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.056671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056626    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-kubernetes\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056682    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kq7z\" (UniqueName: \"kubernetes.io/projected/09bce51b-fc68-45c5-8a20-58f1361b3790-kube-api-access-9kq7z\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-run\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056767    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-kubernetes\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.056906    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.056974    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret podName:6d52fd17-9aa5-4d34-a16e-16220c51a87c nodeName:}" failed. No retries permitted until 2026-09-16 16:36:27.556955944 +0000 UTC m=+3.053829066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret") pod "global-pull-secret-syncer-b2pqs" (UID: "6d52fd17-9aa5-4d34-a16e-16220c51a87c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.056989    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-iptables-alerter-script\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-run-netns\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-cni-bin\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-run\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-serviceca\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-cni-netd\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057207    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-var-lib-kubelet\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057221    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-cni-bin\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysconfig\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-run-netns\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.057553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-cni-netd\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057278    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-sys\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-ovn\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-sys\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-env-overrides\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-ovn\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysctl-conf\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysconfig\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nnx5\" (UniqueName: \"kubernetes.io/projected/83be222b-05e2-44b0-90c5-bf387e175d3e-kube-api-access-8nnx5\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c29mj\" (UniqueName: \"kubernetes.io/projected/6838d84d-a423-44bc-92a2-a57b635f6a51-kube-api-access-c29mj\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-kubelet\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057534    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d52fd17-9aa5-4d34-a16e-16220c51a87c-kubelet-config\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8svxn\" (UniqueName: \"kubernetes.io/projected/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-kube-api-access-8svxn\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-var-lib-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d52fd17-9aa5-4d34-a16e-16220c51a87c-dbus\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-lib-modules\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-systemd\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6838d84d-a423-44bc-92a2-a57b635f6a51-tmp\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.059158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-systemd\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-env-overrides\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-log-socket\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-systemd\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-host-kubelet\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-sysctl-conf\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.057804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-serviceca\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d52fd17-9aa5-4d34-a16e-16220c51a87c-kubelet-config\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-log-socket\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-ovnkube-config\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-lib-modules\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d52fd17-9aa5-4d34-a16e-16220c51a87c-dbus\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-systemd\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058270    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6838d84d-a423-44bc-92a2-a57b635f6a51-etc-tuned\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-var-lib-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.055522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/09bce51b-fc68-45c5-8a20-58f1361b3790-run-openvswitch\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.060158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.058657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/09bce51b-fc68-45c5-8a20-58f1361b3790-ovnkube-config\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.061008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.060718    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6838d84d-a423-44bc-92a2-a57b635f6a51-tmp\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.061891 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.061800    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/09bce51b-fc68-45c5-8a20-58f1361b3790-ovn-node-metrics-cert\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.074128 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.074069    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:27.074128 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.074097    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:27.074128 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.074110    2577 projected.go:196] Error preparing data for projected volume kube-api-access-597d8 for pod openshift-network-diagnostics/network-check-target-rzt9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:27.074339 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.074183    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8 podName:1f444370-24e0-448c-9487-664f0d994dd7 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:27.574165891 +0000 UTC m=+3.071039016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-597d8" (UniqueName: "kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8") pod "network-check-target-rzt9z" (UID: "1f444370-24e0-448c-9487-664f0d994dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:27.076413 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.076389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8svxn\" (UniqueName: \"kubernetes.io/projected/5bacc06b-d05a-4aff-a114-c11a9fdebfb0-kube-api-access-8svxn\") pod \"iptables-alerter-wqq6j\" (UID: \"5bacc06b-d05a-4aff-a114-c11a9fdebfb0\") " pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:27.077133 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.077094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kq7z\" (UniqueName: \"kubernetes.io/projected/09bce51b-fc68-45c5-8a20-58f1361b3790-kube-api-access-9kq7z\") pod \"ovnkube-node-tbzhg\" (UID: \"09bce51b-fc68-45c5-8a20-58f1361b3790\") " pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.077871 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.077852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdq6m\" (UniqueName: \"kubernetes.io/projected/b1d7e520-75a5-4587-b4f3-77ecdf1b2460-kube-api-access-cdq6m\") pod \"node-ca-4ft9d\" (UID: \"b1d7e520-75a5-4587-b4f3-77ecdf1b2460\") " pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:27.083194 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.083170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nnx5\" (UniqueName: \"kubernetes.io/projected/83be222b-05e2-44b0-90c5-bf387e175d3e-kube-api-access-8nnx5\") pod \"node-resolver-htltb\" (UID: \"83be222b-05e2-44b0-90c5-bf387e175d3e\") " pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:27.084209 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.084189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c29mj\" (UniqueName: \"kubernetes.io/projected/6838d84d-a423-44bc-92a2-a57b635f6a51-kube-api-access-c29mj\") pod \"tuned-h9l5b\" (UID: \"6838d84d-a423-44bc-92a2-a57b635f6a51\") " pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.139960 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.139923    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh"
Sep 16 16:36:27.148838 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.148809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p7zhj"
Sep 16 16:36:27.159232 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.159207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:27.166272 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.166249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fpwb6"
Sep 16 16:36:27.173692 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.173671    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h9l5b"
Sep 16 16:36:27.181571 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.181538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-htltb"
Sep 16 16:36:27.190190 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.190108    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ft9d"
Sep 16 16:36:27.198883 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.198861    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wqq6j"
Sep 16 16:36:27.204974 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.204952    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:27.236020 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.235956    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 16:36:27.462411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.462339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:27.462561 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.462456    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:27.462561 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.462514    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:28.462498774 +0000 UTC m=+3.959371877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:27.563645 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.563603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:27.563825 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.563742    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:27.563874 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.563825    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret podName:6d52fd17-9aa5-4d34-a16e-16220c51a87c nodeName:}" failed. No retries permitted until 2026-09-16 16:36:28.563805528 +0000 UTC m=+4.060678653 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret") pod "global-pull-secret-syncer-b2pqs" (UID: "6d52fd17-9aa5-4d34-a16e-16220c51a87c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:27.664631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.664583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:27.664818 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.664768    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:27.664818 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.664796    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:27.664818 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.664810    2577 projected.go:196] Error preparing data for projected volume kube-api-access-597d8 for pod openshift-network-diagnostics/network-check-target-rzt9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:27.664987 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:27.664883    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8 podName:1f444370-24e0-448c-9487-664f0d994dd7 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:28.664863431 +0000 UTC m=+4.161736560 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-597d8" (UniqueName: "kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8") pod "network-check-target-rzt9z" (UID: "1f444370-24e0-448c-9487-664f0d994dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:27.942832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.942741    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 16:31:27 +0000 UTC" deadline="2028-04-24 23:12:28.580898099 +0000 UTC"
Sep 16 16:36:27.942832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:27.942772    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14070h36m0.638130168s"
Sep 16 16:36:28.230221 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:28.230192    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2570fd8_d0e6_4ce4_a62c_65e9b5c22533.slice/crio-4a4904d84e632772e2e1d2e83086836bc5d21abc8ce0f5e1c284e7a29cd37a37 WatchSource:0}: Error finding container 4a4904d84e632772e2e1d2e83086836bc5d21abc8ce0f5e1c284e7a29cd37a37: Status 404 returned error can't find the container with id 4a4904d84e632772e2e1d2e83086836bc5d21abc8ce0f5e1c284e7a29cd37a37
Sep 16 16:36:28.231747 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:28.231572    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bacc06b_d05a_4aff_a114_c11a9fdebfb0.slice/crio-6b4093c425335cb39b820a575f212f3f34f29371a3174a1d984ce87f870c66f2 WatchSource:0}: Error finding container 6b4093c425335cb39b820a575f212f3f34f29371a3174a1d984ce87f870c66f2: Status 404 returned error can't find the container with id 6b4093c425335cb39b820a575f212f3f34f29371a3174a1d984ce87f870c66f2
Sep 16 16:36:28.235336 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:28.235316    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30f3f32_2dd2_4c5c_a9a9_03328722a78f.slice/crio-278c7775d2d85c905d55657d73c0cfb1a513d00bbd445e5b3aa6f019d3a1bf41 WatchSource:0}: Error finding container 278c7775d2d85c905d55657d73c0cfb1a513d00bbd445e5b3aa6f019d3a1bf41: Status 404 returned error can't find the container with id 278c7775d2d85c905d55657d73c0cfb1a513d00bbd445e5b3aa6f019d3a1bf41
Sep 16 16:36:28.236184 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:28.236158    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09bce51b_fc68_45c5_8a20_58f1361b3790.slice/crio-a24a89d63d2b1278a963167fa0a404fe1f23549f7daf633e7d1bfa5ed94e065c WatchSource:0}: Error finding container a24a89d63d2b1278a963167fa0a404fe1f23549f7daf633e7d1bfa5ed94e065c: Status 404 returned error can't find the container with id a24a89d63d2b1278a963167fa0a404fe1f23549f7daf633e7d1bfa5ed94e065c
Sep 16 16:36:28.237097 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:36:28.237076    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83be222b_05e2_44b0_90c5_bf387e175d3e.slice/crio-71ea235d83fea0e1727edd7f4d0f8dc7e11990e8a32041bbbe2d3f9ea23eddaa WatchSource:0}: Error finding container 71ea235d83fea0e1727edd7f4d0f8dc7e11990e8a32041bbbe2d3f9ea23eddaa: Status 404 returned error can't find the container with id 71ea235d83fea0e1727edd7f4d0f8dc7e11990e8a32041bbbe2d3f9ea23eddaa
Sep 16 16:36:28.470966 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:28.470933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:28.471110 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.471057    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:28.471110 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.471107    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:30.471093775 +0000 UTC m=+5.967966893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:28.571828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:28.571791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:28.571943 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.571919    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:28.571988 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.571980    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret podName:6d52fd17-9aa5-4d34-a16e-16220c51a87c nodeName:}" failed. No retries permitted until 2026-09-16 16:36:30.571964182 +0000 UTC m=+6.068837291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret") pod "global-pull-secret-syncer-b2pqs" (UID: "6d52fd17-9aa5-4d34-a16e-16220c51a87c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:28.672213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:28.672177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:28.672387 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.672362    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:28.672476 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.672394    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:28.672476 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.672416    2577 projected.go:196] Error preparing data for projected volume kube-api-access-597d8 for pod openshift-network-diagnostics/network-check-target-rzt9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:28.672605 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.672482    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8 podName:1f444370-24e0-448c-9487-664f0d994dd7 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:30.672457214 +0000 UTC m=+6.169330344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-597d8" (UniqueName: "kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8") pod "network-check-target-rzt9z" (UID: "1f444370-24e0-448c-9487-664f0d994dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:28.943759 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:28.943616    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 16:31:27 +0000 UTC" deadline="2028-05-05 18:11:06.27597982 +0000 UTC"
Sep 16 16:36:28.943759 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:28.943681    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14329h34m37.332304251s"
Sep 16 16:36:28.974930 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:28.973895    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:28.974930 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.974042    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:28.974930 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:28.974473    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:28.974930 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.974570    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:28.974930 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:28.974774    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:28.974930 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:28.974863    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:29.010148 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.010114    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae7a50717792db9bcd51d851252d6e22" containerID="55d1eff5dc713a2021000b137533a3133b1693327cd7b71c0691f4af9479a4e1" exitCode=0
Sep 16 16:36:29.010321 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.010210    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal" event={"ID":"ae7a50717792db9bcd51d851252d6e22","Type":"ContainerDied","Data":"55d1eff5dc713a2021000b137533a3133b1693327cd7b71c0691f4af9479a4e1"}
Sep 16 16:36:29.015840 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.015782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h9l5b" event={"ID":"6838d84d-a423-44bc-92a2-a57b635f6a51","Type":"ContainerStarted","Data":"704836ac191831c76dbb274f5ad289fe58aabd3c637ac4efbd9950ed8064a4ee"}
Sep 16 16:36:29.026302 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.026217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ft9d" event={"ID":"b1d7e520-75a5-4587-b4f3-77ecdf1b2460","Type":"ContainerStarted","Data":"f2b62d37820050477c1c6b9855121154dd08a2cba6337bf5c957e5ec8b67098f"}
Sep 16 16:36:29.030012 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.029934    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh" event={"ID":"b14f8e9c-19ed-4cda-b9f2-91142a85624a","Type":"ContainerStarted","Data":"9910fc1dfd950dde9655841cbcb6b70bfd86d77b7eead231f4c22c500365bef2"}
Sep 16 16:36:29.032786 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.032682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g4mhj" event={"ID":"e30f3f32-2dd2-4c5c-a9a9-03328722a78f","Type":"ContainerStarted","Data":"278c7775d2d85c905d55657d73c0cfb1a513d00bbd445e5b3aa6f019d3a1bf41"}
Sep 16 16:36:29.045919 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.045861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wqq6j" event={"ID":"5bacc06b-d05a-4aff-a114-c11a9fdebfb0","Type":"ContainerStarted","Data":"6b4093c425335cb39b820a575f212f3f34f29371a3174a1d984ce87f870c66f2"}
Sep 16 16:36:29.053662 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.053534    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"a24a89d63d2b1278a963167fa0a404fe1f23549f7daf633e7d1bfa5ed94e065c"}
Sep 16 16:36:29.064179 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.064101    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fpwb6" event={"ID":"6ebfc096-7519-4787-a95a-b39896f1123d","Type":"ContainerStarted","Data":"80083fc8fe7f77f755f0d0cdef3e55ae724a7944878950dc6bb61d4bd0cb9cbb"}
Sep 16 16:36:29.075921 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.075867    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-htltb" event={"ID":"83be222b-05e2-44b0-90c5-bf387e175d3e","Type":"ContainerStarted","Data":"71ea235d83fea0e1727edd7f4d0f8dc7e11990e8a32041bbbe2d3f9ea23eddaa"}
Sep 16 16:36:29.092450 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.092421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" event={"ID":"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533","Type":"ContainerStarted","Data":"4a4904d84e632772e2e1d2e83086836bc5d21abc8ce0f5e1c284e7a29cd37a37"}
Sep 16 16:36:29.102205 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.102177    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal" event={"ID":"29c4f9a3e72c82af2027e396c27340f6","Type":"ContainerStarted","Data":"a23245d1e6e8d3c62b6f593c5f91c791a480a669a86db57884c286e753913514"}
Sep 16 16:36:29.124034 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:29.123905    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-51.ec2.internal" podStartSLOduration=3.123886819 podStartE2EDuration="3.123886819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:29.122350348 +0000 UTC m=+4.619223478" watchObservedRunningTime="2026-09-16 16:36:29.123886819 +0000 UTC m=+4.620759947"
Sep 16 16:36:30.122029 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:30.121690    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal" event={"ID":"ae7a50717792db9bcd51d851252d6e22","Type":"ContainerStarted","Data":"563c47ce547c14a7cabc1e259d1df7dead83aef95c87a0bcf184356c0ca0e394"}
Sep 16 16:36:30.144964 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:30.144879    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-51.ec2.internal" podStartSLOduration=4.144858018 podStartE2EDuration="4.144858018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:30.144441092 +0000 UTC m=+5.641314221" watchObservedRunningTime="2026-09-16 16:36:30.144858018 +0000 UTC m=+5.641731146"
Sep 16 16:36:30.493711 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:30.492864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:30.493711 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.493088    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:30.493711 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.493156    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:34.493137232 +0000 UTC m=+9.990010354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:30.595025 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:30.594361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:30.595025 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.594606    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:30.595025 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.594682    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret podName:6d52fd17-9aa5-4d34-a16e-16220c51a87c nodeName:}" failed. No retries permitted until 2026-09-16 16:36:34.594661601 +0000 UTC m=+10.091534732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret") pod "global-pull-secret-syncer-b2pqs" (UID: "6d52fd17-9aa5-4d34-a16e-16220c51a87c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:30.694977 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:30.694941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:30.695190 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.695149    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:30.695190 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.695171    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:30.695190 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.695183    2577 projected.go:196] Error preparing data for projected volume kube-api-access-597d8 for pod openshift-network-diagnostics/network-check-target-rzt9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:30.695387 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.695238    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8 podName:1f444370-24e0-448c-9487-664f0d994dd7 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:34.695224953 +0000 UTC m=+10.192098056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-597d8" (UniqueName: "kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8") pod "network-check-target-rzt9z" (UID: "1f444370-24e0-448c-9487-664f0d994dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:30.972494 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:30.971730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:30.972494 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.971880    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:30.972494 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:30.972283    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:30.972494 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.972382    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:30.973106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:30.972941    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:30.973106 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:30.973058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:32.971539 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:32.971500    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:32.972042 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:32.971692    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:32.972460 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:32.972154    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:32.972460 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:32.972304    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:32.972460 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:32.972341    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:32.972460 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:32.972428    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:34.528318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:34.528286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:34.528710 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.528456    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:34.528710 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.528515    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:42.528498032 +0000 UTC m=+18.025371141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:34.629657 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:34.629619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:34.629835 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.629775    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:34.629898 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.629835    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret podName:6d52fd17-9aa5-4d34-a16e-16220c51a87c nodeName:}" failed. No retries permitted until 2026-09-16 16:36:42.629816796 +0000 UTC m=+18.126689915 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret") pod "global-pull-secret-syncer-b2pqs" (UID: "6d52fd17-9aa5-4d34-a16e-16220c51a87c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:34.731636 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:34.731039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:34.731636 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.731185    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:34.731636 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.731208    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:34.731636 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.731220    2577 projected.go:196] Error preparing data for projected volume kube-api-access-597d8 for pod openshift-network-diagnostics/network-check-target-rzt9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:34.731636 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.731274    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8 podName:1f444370-24e0-448c-9487-664f0d994dd7 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:42.731256315 +0000 UTC m=+18.228129423 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-597d8" (UniqueName: "kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8") pod "network-check-target-rzt9z" (UID: "1f444370-24e0-448c-9487-664f0d994dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:34.972283 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:34.972214    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:34.972427 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:34.972307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:34.972427 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.972323    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:34.972555 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.972446    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:34.972555 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:34.972494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:34.972643 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:34.972579    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:36.971902 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:36.971867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:36.972297 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:36.971870    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:36.972297 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:36.972014    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:36.972297 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:36.972037    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:36.972297 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:36.972148    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:36.972297 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:36.972252    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:38.971229 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:38.971207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:38.971611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:38.971213    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:38.971611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:38.971322    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:38.971611 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:38.971319    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:38.971611 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:38.971374    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:38.971611 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:38.971464    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:40.970986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:40.970950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:40.971452 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:40.970992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:40.971452 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:40.970950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:40.971452 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:40.971084    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:40.971452 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:40.971178    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:40.971452 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:40.971270    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:42.591631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:42.591380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:42.591631 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.591559    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:42.591631 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.591623    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:58.591603598 +0000 UTC m=+34.088476715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:42.692159 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:42.692085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:42.692315 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.692242    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:42.692315 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.692314    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret podName:6d52fd17-9aa5-4d34-a16e-16220c51a87c nodeName:}" failed. No retries permitted until 2026-09-16 16:36:58.692296119 +0000 UTC m=+34.189169227 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret") pod "global-pull-secret-syncer-b2pqs" (UID: "6d52fd17-9aa5-4d34-a16e-16220c51a87c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:42.792690 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:42.792655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:42.792860 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.792840    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:42.792925 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.792861    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:42.792925 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.792871    2577 projected.go:196] Error preparing data for projected volume kube-api-access-597d8 for pod openshift-network-diagnostics/network-check-target-rzt9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:42.792925 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.792923    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8 podName:1f444370-24e0-448c-9487-664f0d994dd7 nodeName:}" failed. No retries permitted until 2026-09-16 16:36:58.792910617 +0000 UTC m=+34.289783718 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-597d8" (UniqueName: "kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8") pod "network-check-target-rzt9z" (UID: "1f444370-24e0-448c-9487-664f0d994dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:42.971091 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:42.971017    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:42.971243 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:42.971015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:42.971243 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.971149    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:42.971369 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.971235    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:42.971369 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:42.971015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:42.971369 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:42.971324    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:44.972015 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:44.971978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:44.972425 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:44.972075    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:44.972425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:44.972160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:44.972425 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:44.972248    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:44.972425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:44.972293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:44.972425 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:44.972352    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:46.971545 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:46.971182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:46.972188 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:46.971598    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:46.972188 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:46.971620    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:46.972188 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:46.971629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:46.972188 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:46.971711    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:46.972188 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:46.971777    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:47.160320 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.160281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh" event={"ID":"b14f8e9c-19ed-4cda-b9f2-91142a85624a","Type":"ContainerStarted","Data":"e96443ff06385e4a85217a8262cd9b7589bf8616583cbe2241793b3366da3794"}
Sep 16 16:36:47.161994 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.161954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g4mhj" event={"ID":"e30f3f32-2dd2-4c5c-a9a9-03328722a78f","Type":"ContainerStarted","Data":"b8d06e7df2f97dc25726710ed886128de79ee8f0d96c9762f59eb4340f8f49ab"}
Sep 16 16:36:47.165458 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.165419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"1c5a64c2bfc52f0c20becd5babdf2123bd2eef63d1f8559f8afc020a18286047"}
Sep 16 16:36:47.165604 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.165521    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"26ddb1227da5c37a4c06fcbb1cf55204fb7898f819f210e30ddbc16f3dfbeb8d"}
Sep 16 16:36:47.165604 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.165575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"280114885e6596881b3b13c672b0e4041878ee545391984599d65593691551b6"}
Sep 16 16:36:47.165731 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.165622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"3f94258f4d64116b94078317108b30cdedabb335e2999923edaf2732b9aecf84"}
Sep 16 16:36:47.165731 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.165660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"1b96ee6f579ec5174113f757f69401c463e04be4dee28d3677e7cbf889bbdf87"}
Sep 16 16:36:47.165731 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.165688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"85693c38a9391f133ff0bcdf86e10618238fc0706781734a7376e0696a4a68d1"}
Sep 16 16:36:47.167106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.167078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fpwb6" event={"ID":"6ebfc096-7519-4787-a95a-b39896f1123d","Type":"ContainerStarted","Data":"f73204d1acd71c1e49181ed75df370263fa08b92faf561498a0351f41d31cc23"}
Sep 16 16:36:47.168789 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.168765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-htltb" event={"ID":"83be222b-05e2-44b0-90c5-bf387e175d3e","Type":"ContainerStarted","Data":"e76569060664de3cbeb85eca62154c0ab023f30e3f891774904b40018661d77e"}
Sep 16 16:36:47.170518 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.170490    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2570fd8-d0e6-4ce4-a62c-65e9b5c22533" containerID="f5e999c5ff4b50301a8b75f38cdd7446a11cdc4899597fcc3ca947d04aeec89c" exitCode=0
Sep 16 16:36:47.170633 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.170609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" event={"ID":"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533","Type":"ContainerDied","Data":"f5e999c5ff4b50301a8b75f38cdd7446a11cdc4899597fcc3ca947d04aeec89c"}
Sep 16 16:36:47.172482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.172460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h9l5b" event={"ID":"6838d84d-a423-44bc-92a2-a57b635f6a51","Type":"ContainerStarted","Data":"d76ded8a0541889593ca1adb19775444ba7096d62df498160f400a56c7d91c20"}
Sep 16 16:36:47.174140 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.174122    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ft9d" event={"ID":"b1d7e520-75a5-4587-b4f3-77ecdf1b2460","Type":"ContainerStarted","Data":"e15ff7fb96a4579eda530c1e3c2d6265e0e56eeef848ed6e1460413df94f04cb"}
Sep 16 16:36:47.190641 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.190594    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g4mhj" podStartSLOduration=8.067162291 podStartE2EDuration="22.190581549s" totalImagesPullingTime="14.123419258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:47.189226599 +0000 UTC m=+22.686099724" watchObservedRunningTime="2026-09-16 16:36:47.190581549 +0000 UTC m=+22.687454664"
Sep 16 16:36:47.282101 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.282039    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4ft9d" podStartSLOduration=4.320241769 podStartE2EDuration="22.282023374s" totalImagesPullingTime="17.961781605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:47.253139374 +0000 UTC m=+22.750012501" watchObservedRunningTime="2026-09-16 16:36:47.282023374 +0000 UTC m=+22.778896498"
Sep 16 16:36:47.305926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.305883    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fpwb6" podStartSLOduration=4.283259008 podStartE2EDuration="22.305867793s" totalImagesPullingTime="18.022608785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:47.283312945 +0000 UTC m=+22.780186070" watchObservedRunningTime="2026-09-16 16:36:47.305867793 +0000 UTC m=+22.802740966"
Sep 16 16:36:47.312605 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.312581    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 16:36:47.336266 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.336230    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h9l5b" podStartSLOduration=4.372365292 podStartE2EDuration="22.336217099s" totalImagesPullingTime="17.963851807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:47.306307883 +0000 UTC m=+22.803181009" watchObservedRunningTime="2026-09-16 16:36:47.336217099 +0000 UTC m=+22.833090222"
Sep 16 16:36:47.336353 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.336311    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-htltb" podStartSLOduration=4.3725305070000005 podStartE2EDuration="22.336307495s" totalImagesPullingTime="17.963776988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:47.336097249 +0000 UTC m=+22.832970373" watchObservedRunningTime="2026-09-16 16:36:47.336307495 +0000 UTC m=+22.833180618"
Sep 16 16:36:47.896777 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.896648    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T16:36:47.312601547Z","UUID":"3570b4ab-726e-4800-a427-7316e5d97ec3","Handler":null,"Name":"","Endpoint":""}
Sep 16 16:36:47.898090 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.898068    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 16:36:47.898207 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:47.898100    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 16:36:48.179504 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:48.179432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh" event={"ID":"b14f8e9c-19ed-4cda-b9f2-91142a85624a","Type":"ContainerStarted","Data":"f898712a570d7e1e1513b465ebb3ba7cf2f31c9db88f5d533191f348c3367451"}
Sep 16 16:36:48.181213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:48.181186    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wqq6j" event={"ID":"5bacc06b-d05a-4aff-a114-c11a9fdebfb0","Type":"ContainerStarted","Data":"4ccb4931a370183f0080466e162dd89f598c389e4c7fe8113ec506a5a3a5d48c"}
Sep 16 16:36:48.971559 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:48.971517    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:48.971695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:48.971520    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:48.971790 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:48.971670    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:48.972054 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:48.971905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:48.972054 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:48.971875    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:48.972054 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:48.972017    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:49.188014 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:49.187893    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh" event={"ID":"b14f8e9c-19ed-4cda-b9f2-91142a85624a","Type":"ContainerStarted","Data":"165eb2c92e12a3a4b37ff3403e377f844d91e33b7973e94b4cb9e2ab428b3b63"}
Sep 16 16:36:49.192960 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:49.192926    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"d6d61f8a9473fedc2dc8815249c3aebac8c1b47bb6e780a396352fe6a6eb3a59"}
Sep 16 16:36:49.217226 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:49.217183    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wqq6j" podStartSLOduration=6.247146957 podStartE2EDuration="24.217169036s" totalImagesPullingTime="17.970022079s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:48.199233094 +0000 UTC m=+23.696106218" watchObservedRunningTime="2026-09-16 16:36:49.217169036 +0000 UTC m=+24.714042160"
Sep 16 16:36:49.217839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:49.217805    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fcfrh" podStartSLOduration=4.404778237 podStartE2EDuration="24.21779359s" totalImagesPullingTime="19.813015353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:49.217585142 +0000 UTC m=+24.714458267" watchObservedRunningTime="2026-09-16 16:36:49.21779359 +0000 UTC m=+24.714666715"
Sep 16 16:36:50.461518 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:50.461485    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:50.462193 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:50.462174    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:50.970949 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:50.970915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:50.971119 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:50.970914    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:50.971119 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:50.971036    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:50.971239 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:50.971150    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:50.971239 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:50.970915    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:50.971340 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:50.971257    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:52.201450 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.201419    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2570fd8-d0e6-4ce4-a62c-65e9b5c22533" containerID="ec275cbb5104a7bd6bc093b03a4c84881697e4e5c7b1661a273afd9240eeb2ff" exitCode=0
Sep 16 16:36:52.202238 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.201511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" event={"ID":"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533","Type":"ContainerDied","Data":"ec275cbb5104a7bd6bc093b03a4c84881697e4e5c7b1661a273afd9240eeb2ff"}
Sep 16 16:36:52.205426 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.205408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" event={"ID":"09bce51b-fc68-45c5-8a20-58f1361b3790","Type":"ContainerStarted","Data":"79a81f5248e951e437da08a1333e3c06a3e151882bbf14f59c245693e2618849"}
Sep 16 16:36:52.205717 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.205702    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:52.205798 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.205743    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:52.220774 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.220754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:52.251437 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.251397    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg" podStartSLOduration=9.224476122 podStartE2EDuration="27.251384736s" totalImagesPullingTime="18.026908614s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:36:52.249895985 +0000 UTC m=+27.746769103" watchObservedRunningTime="2026-09-16 16:36:52.251384736 +0000 UTC m=+27.748257860"
Sep 16 16:36:52.971855 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.971618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:52.972115 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:52.971960    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:52.972115 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.971756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:52.972224 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:52.972135    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:52.972224 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:52.971618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:52.972326 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:52.972257    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:53.207836 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.207801    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:53.221952 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.221888    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:36:53.246539 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.246496    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b2pqs"]
Sep 16 16:36:53.246640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.246613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:53.246745 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:53.246722    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:53.250153 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.250128    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9wkss"]
Sep 16 16:36:53.250267 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.250217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:53.250330 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:53.250303    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:53.252373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.252349    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rzt9z"]
Sep 16 16:36:53.252474 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.252413    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:53.252556 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:53.252475    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:53.261062 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.261041    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:53.261156 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.261148    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 16:36:53.261598 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:53.261583    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g4mhj"
Sep 16 16:36:54.211522 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:54.211487    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2570fd8-d0e6-4ce4-a62c-65e9b5c22533" containerID="684e1ba429238730015ec637ff15450ac9e31bd4f87c92f16b4e7cfb876f7d9d" exitCode=0
Sep 16 16:36:54.212001 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:54.211801    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" event={"ID":"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533","Type":"ContainerDied","Data":"684e1ba429238730015ec637ff15450ac9e31bd4f87c92f16b4e7cfb876f7d9d"}
Sep 16 16:36:54.972238 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:54.972207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:54.972422 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:54.972297    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:54.972422 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:54.972390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:54.972556 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:54.972485    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:54.972611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:54.972558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:54.972696 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:54.972677    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:56.218185 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:56.218023    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2570fd8-d0e6-4ce4-a62c-65e9b5c22533" containerID="f57399a4ae4299ae4094231bc042af92e00c5eb7889b0f394853f86a95fb3a2c" exitCode=0
Sep 16 16:36:56.218185 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:56.218097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" event={"ID":"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533","Type":"ContainerDied","Data":"f57399a4ae4299ae4094231bc042af92e00c5eb7889b0f394853f86a95fb3a2c"}
Sep 16 16:36:56.972002 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:56.971724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:56.972002 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:56.971772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:56.972002 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:56.971724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:56.972002 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:56.971865    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:56.972002 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:56.971935    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:56.972388 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:56.972014    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:57.210136 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:57.210097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:36:57.345146 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:57.345054    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:36:58.609485 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:58.609452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:58.610168 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.609616    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:58.610168 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.609668    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:30.609655847 +0000 UTC m=+66.106528962 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 16:36:58.710667 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:58.710629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:58.710864 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.710785    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:58.710864 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.710861    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret podName:6d52fd17-9aa5-4d34-a16e-16220c51a87c nodeName:}" failed. No retries permitted until 2026-09-16 16:37:30.710842431 +0000 UTC m=+66.207715537 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret") pod "global-pull-secret-syncer-b2pqs" (UID: "6d52fd17-9aa5-4d34-a16e-16220c51a87c") : object "kube-system"/"original-pull-secret" not registered
Sep 16 16:36:58.810964 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:58.810932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:58.811135 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.811113    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 16:36:58.811135 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.811133    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 16:36:58.811214 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.811142    2577 projected.go:196] Error preparing data for projected volume kube-api-access-597d8 for pod openshift-network-diagnostics/network-check-target-rzt9z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:58.811214 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.811198    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8 podName:1f444370-24e0-448c-9487-664f0d994dd7 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:30.811183031 +0000 UTC m=+66.308056151 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-597d8" (UniqueName: "kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8") pod "network-check-target-rzt9z" (UID: "1f444370-24e0-448c-9487-664f0d994dd7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 16:36:58.971572 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:58.971489    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:36:58.971722 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:58.971487    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:36:58.971722 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.971621    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rzt9z" podUID="1f444370-24e0-448c-9487-664f0d994dd7"
Sep 16 16:36:58.971722 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:58.971676    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:36:58.971852 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.971769    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:36:58.971852 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:58.971838    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-b2pqs" podUID="6d52fd17-9aa5-4d34-a16e-16220c51a87c"
Sep 16 16:36:59.218577 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.218549    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-51.ec2.internal" event="NodeReady"
Sep 16 16:36:59.218737 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.218691    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 16:36:59.254098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.254024    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6788cc6fc8-6n9vv"]
Sep 16 16:36:59.258802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.258781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.261277 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.261249    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 16:36:59.261408 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.261307    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 16:36:59.261408 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.261377    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 16:36:59.261555 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.261523    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sm9dz\""
Sep 16 16:36:59.270391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.270367    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b5bfg"]
Sep 16 16:36:59.272053 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.272034    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 16:36:59.274666 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.274631    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6788cc6fc8-6n9vv"]
Sep 16 16:36:59.274765 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.274745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.275303 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.275285    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rds89"]
Sep 16 16:36:59.277304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.277286    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 16:36:59.277404 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.277290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bqrg9\""
Sep 16 16:36:59.277654 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.277637    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 16:36:59.278350 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.278333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rds89"
Sep 16 16:36:59.281128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.281107    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 16:36:59.281247 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.281162    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6pkw2\""
Sep 16 16:36:59.281611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.281588    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 16:36:59.281731 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.281713    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 16:36:59.288122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.288105    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b5bfg"]
Sep 16 16:36:59.292920 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.292900    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rds89"]
Sep 16 16:36:59.416456 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-image-registry-private-configuration\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.416631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4423a1a4-565b-402b-a813-b228a1d2213c-cert\") pod \"ingress-canary-rds89\" (UID: \"4423a1a4-565b-402b-a813-b228a1d2213c\") " pod="openshift-ingress-canary/ingress-canary-rds89"
Sep 16 16:36:59.416631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34007bd9-3504-45a4-9175-3582bffd83d6-ca-trust-extracted\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.416631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-trusted-ca\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.416767 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmffp\" (UniqueName: \"kubernetes.io/projected/4423a1a4-565b-402b-a813-b228a1d2213c-kube-api-access-nmffp\") pod \"ingress-canary-rds89\" (UID: \"4423a1a4-565b-402b-a813-b228a1d2213c\") " pod="openshift-ingress-canary/ingress-canary-rds89"
Sep 16 16:36:59.416767 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-config-volume\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.416767 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqt5f\" (UniqueName: \"kubernetes.io/projected/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-kube-api-access-bqt5f\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.416904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.416904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.416904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-bound-sa-token\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.416904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-installation-pull-secrets\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.416904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5tsx\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-kube-api-access-d5tsx\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.417085 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416921    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-tmp-dir\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.417085 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.416949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-registry-certificates\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.446422 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.446381    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:36:59.517551 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.517551 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.517551 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-bound-sa-token\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-installation-pull-secrets\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5tsx\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-kube-api-access-d5tsx\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-tmp-dir\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:59.517640    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:59.517707    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:37:00.017688172 +0000 UTC m=+35.514561288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:59.517758    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:59.517770    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-registry-certificates\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.517820 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:36:59.517803    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:00.017792959 +0000 UTC m=+35.514666062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:36:59.518191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-image-registry-private-configuration\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.518191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4423a1a4-565b-402b-a813-b228a1d2213c-cert\") pod \"ingress-canary-rds89\" (UID: \"4423a1a4-565b-402b-a813-b228a1d2213c\") " pod="openshift-ingress-canary/ingress-canary-rds89"
Sep 16 16:36:59.518191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34007bd9-3504-45a4-9175-3582bffd83d6-ca-trust-extracted\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.518191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517948    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-tmp-dir\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.518191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.517958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-trusted-ca\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.518191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.518010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmffp\" (UniqueName: \"kubernetes.io/projected/4423a1a4-565b-402b-a813-b228a1d2213c-kube-api-access-nmffp\") pod \"ingress-canary-rds89\" (UID: \"4423a1a4-565b-402b-a813-b228a1d2213c\") " pod="openshift-ingress-canary/ingress-canary-rds89"
Sep 16 16:36:59.518191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.518105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-config-volume\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.518191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.518149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqt5f\" (UniqueName: \"kubernetes.io/projected/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-kube-api-access-bqt5f\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.518586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.518515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34007bd9-3504-45a4-9175-3582bffd83d6-ca-trust-extracted\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.518690 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.518668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-registry-certificates\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.519076 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.519022    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-config-volume\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.519698 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.519676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-trusted-ca\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.522434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.522415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-installation-pull-secrets\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.522543 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.522438    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-image-registry-private-configuration\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.522543 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.522481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4423a1a4-565b-402b-a813-b228a1d2213c-cert\") pod \"ingress-canary-rds89\" (UID: \"4423a1a4-565b-402b-a813-b228a1d2213c\") " pod="openshift-ingress-canary/ingress-canary-rds89"
Sep 16 16:36:59.529879 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.529854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-bound-sa-token\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.532278 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.532256    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmffp\" (UniqueName: \"kubernetes.io/projected/4423a1a4-565b-402b-a813-b228a1d2213c-kube-api-access-nmffp\") pod \"ingress-canary-rds89\" (UID: \"4423a1a4-565b-402b-a813-b228a1d2213c\") " pod="openshift-ingress-canary/ingress-canary-rds89"
Sep 16 16:36:59.532574 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.532549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqt5f\" (UniqueName: \"kubernetes.io/projected/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-kube-api-access-bqt5f\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:36:59.532925 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.532901    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5tsx\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-kube-api-access-d5tsx\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:36:59.580833 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.580771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:36:59.592329 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:36:59.592308    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rds89"
Sep 16 16:37:00.023593 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.023563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:37:00.024718 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.023606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:37:00.024718 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:00.023775    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:00.024718 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:00.023780    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:00.024718 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:00.023791    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:37:00.024718 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:00.023874    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:37:01.023851713 +0000 UTC m=+36.520724822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:00.024718 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:00.023925    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:01.023906166 +0000 UTC m=+36.520779284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:37:00.229199 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.229118    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2570fd8-d0e6-4ce4-a62c-65e9b5c22533" containerID="18ab314cecd67606c2082ceb7738911a0f7a935f0accd44a0f454126c2020b34" exitCode=0
Sep 16 16:37:00.229199 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.229154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" event={"ID":"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533","Type":"ContainerDied","Data":"18ab314cecd67606c2082ceb7738911a0f7a935f0accd44a0f454126c2020b34"}
Sep 16 16:37:00.236955 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.236934    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rds89"]
Sep 16 16:37:00.240439 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:37:00.240415    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4423a1a4_565b_402b_a813_b228a1d2213c.slice/crio-6bd71863c8fef4121446ea01d8e88430f846a2511ead76758e1dec804990aa8d WatchSource:0}: Error finding container 6bd71863c8fef4121446ea01d8e88430f846a2511ead76758e1dec804990aa8d: Status 404 returned error can't find the container with id 6bd71863c8fef4121446ea01d8e88430f846a2511ead76758e1dec804990aa8d
Sep 16 16:37:00.971083 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.971046    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:37:00.971083 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.971046    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:37:00.971418 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.971393    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:37:00.977957 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.977927    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 16:37:00.978721 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.978699    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 16:37:00.978941 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.978926    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtfpq\""
Sep 16 16:37:00.979033 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.979001    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 16:37:00.979110 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.978925    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 16:37:00.979417 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:00.979241    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v6lts\""
Sep 16 16:37:01.030632 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:01.030609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:37:01.030955 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:01.030641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:37:01.030955 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:01.030730    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:01.030955 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:01.030743    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:01.030955 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:01.030762    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:37:01.030955 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:01.030817    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:37:03.030795984 +0000 UTC m=+38.527669092 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:01.030955 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:01.030836    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:03.0308279 +0000 UTC m=+38.527701002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:37:01.162790 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:01.162750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:01.235678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:01.235597    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2570fd8-d0e6-4ce4-a62c-65e9b5c22533" containerID="053aa13472a38f108b5d2eee2ccda0c76c3bb5f924547614bf9a08e6f3e194f6" exitCode=0
Sep 16 16:37:01.235847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:01.235677    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" event={"ID":"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533","Type":"ContainerDied","Data":"053aa13472a38f108b5d2eee2ccda0c76c3bb5f924547614bf9a08e6f3e194f6"}
Sep 16 16:37:01.237139 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:01.237109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rds89" event={"ID":"4423a1a4-565b-402b-a813-b228a1d2213c","Type":"ContainerStarted","Data":"6bd71863c8fef4121446ea01d8e88430f846a2511ead76758e1dec804990aa8d"}
Sep 16 16:37:01.282270 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:01.282239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:02.039607 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.039576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:02.113811 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.113772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:02.212166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.212131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:02.244394 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.244362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" event={"ID":"d2570fd8-d0e6-4ce4-a62c-65e9b5c22533","Type":"ContainerStarted","Data":"8ba774f1fea4a91314b54ad5788e3358dcbabcb5311e94eda7d596371deefe22"}
Sep 16 16:37:02.273126 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.273068    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p7zhj" podStartSLOduration=9.046551617 podStartE2EDuration="37.273045295s" totalImagesPullingTime="28.226493678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:02.271916896 +0000 UTC m=+37.768790031" watchObservedRunningTime="2026-09-16 16:37:02.273045295 +0000 UTC m=+37.769918420"
Sep 16 16:37:02.288660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.288625    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:02.417430 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.417355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:02.519910 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.519877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:02.588472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:02.588434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:03.046549 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:03.046454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:37:03.046549 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:03.046490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:37:03.046926 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:03.046624    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:03.046926 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:03.046691    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:37:07.04667467 +0000 UTC m=+42.543547771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:03.046926 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:03.046633    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:03.046926 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:03.046734    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:37:03.046926 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:03.046784    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:07.046768628 +0000 UTC m=+42.543641729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:37:03.136344 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:03.136305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:03.248548 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:03.248315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rds89" event={"ID":"4423a1a4-565b-402b-a813-b228a1d2213c","Type":"ContainerStarted","Data":"267c4d3f5f53c7a2d04ac728ba7e82c83a8a7657797ee9765b57c907fe53ad04"}
Sep 16 16:37:03.581029 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:03.580999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:03.677166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:03.677137    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:04.377947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:04.377905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:04.571906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:04.571876    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:04.827863 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:04.827813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:04.945340 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:04.945297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:07.071409 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:07.071372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:37:07.071409 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:07.071408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:37:07.071889 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:07.071824    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:07.071938 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:07.071912    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:37:15.071879922 +0000 UTC m=+50.568753024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:07.071938 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:07.071915    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:07.071938 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:07.071937    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:37:07.072083 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:07.072006    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:15.071988889 +0000 UTC m=+50.568861991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:37:12.095337 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:12.095301    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:14.201070 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:14.201024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:14.300666 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:14.300625    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:15.127959 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:15.127918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:37:15.127959 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:15.127961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:37:15.128163 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:15.128074    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:15.128163 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:15.128078    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:15.128163 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:15.128142    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:37:31.128126277 +0000 UTC m=+66.624999382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:15.128266 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:15.128084    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:37:15.128266 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:15.128225    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:37:31.128213514 +0000 UTC m=+66.625086616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:37:18.195563 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:18.195510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:18.291043 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:18.291009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:19.634938 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:19.634897    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:19.702823 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:19.702791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:23.708525 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:23.708476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:23.799801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:23.799775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:25.224379 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:25.224344    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tbzhg"
Sep 16 16:37:25.254845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:25.254797    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rds89" podStartSLOduration=23.819760092 podStartE2EDuration="26.25477663s" totalImagesPullingTime="2.435016538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:03.269914024 +0000 UTC m=+38.766787148" watchObservedRunningTime="2026-09-16 16:37:25.25477663 +0000 UTC m=+60.751649755"
Sep 16 16:37:25.380253 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:25.380214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:25.454142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:25.454107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:30.633588 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.633545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:37:30.636432 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.636412    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 16:37:30.644798 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:30.644778    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 16:37:30.644913 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:30.644854    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:38:34.644831537 +0000 UTC m=+130.141704640 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : secret "metrics-daemon-secret" not found
Sep 16 16:37:30.733890 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.733860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:37:30.736582 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.736566    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 16:37:30.747616 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.747594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d52fd17-9aa5-4d34-a16e-16220c51a87c-original-pull-secret\") pod \"global-pull-secret-syncer-b2pqs\" (UID: \"6d52fd17-9aa5-4d34-a16e-16220c51a87c\") " pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:37:30.834956 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.834915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:37:30.837627 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.837609    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 16:37:30.847691 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.847671    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 16:37:30.858039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.858014    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-597d8\" (UniqueName: \"kubernetes.io/projected/1f444370-24e0-448c-9487-664f0d994dd7-kube-api-access-597d8\") pod \"network-check-target-rzt9z\" (UID: \"1f444370-24e0-448c-9487-664f0d994dd7\") " pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:37:30.994323 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.994300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jtfpq\""
Sep 16 16:37:30.997470 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:30.997456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-b2pqs"
Sep 16 16:37:31.002593 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:31.002569    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:37:31.136966 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:31.136912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:37:31.136966 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:31.136957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:37:31.137124 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:31.137051    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:37:31.137124 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:31.137058    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:37:31.137124 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:31.137071    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:37:31.137124 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:31.137126    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:38:03.137111499 +0000 UTC m=+98.633984621 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:37:31.137314 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:37:31.137140    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:38:03.137134922 +0000 UTC m=+98.634008024 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:37:31.304171 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:31.304095    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-b2pqs"]
Sep 16 16:37:31.307408 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:37:31.307381    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d52fd17_9aa5_4d34_a16e_16220c51a87c.slice/crio-088727c55ab19e13ddb894da73a7bf5337f9c7f3e9d4b3a4c6558c9bdf98f55e WatchSource:0}: Error finding container 088727c55ab19e13ddb894da73a7bf5337f9c7f3e9d4b3a4c6558c9bdf98f55e: Status 404 returned error can't find the container with id 088727c55ab19e13ddb894da73a7bf5337f9c7f3e9d4b3a4c6558c9bdf98f55e
Sep 16 16:37:31.313632 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:31.313609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b2pqs" event={"ID":"6d52fd17-9aa5-4d34-a16e-16220c51a87c","Type":"ContainerStarted","Data":"088727c55ab19e13ddb894da73a7bf5337f9c7f3e9d4b3a4c6558c9bdf98f55e"}
Sep 16 16:37:31.327464 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:31.327444    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rzt9z"]
Sep 16 16:37:31.332262 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:37:31.332199    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f444370_24e0_448c_9487_664f0d994dd7.slice/crio-83bf0a6e283b80f6bb0842ef161775654484e5330ca921471615ab0e2ed0c1c1 WatchSource:0}: Error finding container 83bf0a6e283b80f6bb0842ef161775654484e5330ca921471615ab0e2ed0c1c1: Status 404 returned error can't find the container with id 83bf0a6e283b80f6bb0842ef161775654484e5330ca921471615ab0e2ed0c1c1
Sep 16 16:37:32.318592 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:32.318506    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rzt9z" event={"ID":"1f444370-24e0-448c-9487-664f0d994dd7","Type":"ContainerStarted","Data":"83bf0a6e283b80f6bb0842ef161775654484e5330ca921471615ab0e2ed0c1c1"}
Sep 16 16:37:36.330885 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:36.330843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-b2pqs" event={"ID":"6d52fd17-9aa5-4d34-a16e-16220c51a87c","Type":"ContainerStarted","Data":"345576b0b317174e9e89f881d3ebfcfcba9487958d2048f23da03c06aae74bee"}
Sep 16 16:37:36.332269 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:36.332245    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rzt9z" event={"ID":"1f444370-24e0-448c-9487-664f0d994dd7","Type":"ContainerStarted","Data":"424cd5aa2b36cf078aa7aaa049d0d8d625f0a4dc56830c05a20f7659c3765aa3"}
Sep 16 16:37:36.332384 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:36.332367    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:37:36.347087 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:36.347045    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-b2pqs" podStartSLOduration=67.014127733 podStartE2EDuration="1m11.347029208s" totalImagesPullingTime="4.332901475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:36.346442886 +0000 UTC m=+71.843316009" watchObservedRunningTime="2026-09-16 16:37:36.347029208 +0000 UTC m=+71.843902333"
Sep 16 16:37:36.360951 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:36.360910    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rzt9z" podStartSLOduration=67.055626458 podStartE2EDuration="1m11.360896024s" totalImagesPullingTime="4.305269566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:37:36.360467921 +0000 UTC m=+71.857341045" watchObservedRunningTime="2026-09-16 16:37:36.360896024 +0000 UTC m=+71.857769153"
Sep 16 16:37:40.204167 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:40.204126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:37:40.323259 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:37:40.323230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:38:03.154766 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:38:03.154734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:38:03.155173 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:38:03.154775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:38:03.155173 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:38:03.154885    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:38:03.155173 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:38:03.154887    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:38:03.155173 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:38:03.154963    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:39:07.154949939 +0000 UTC m=+162.651823041 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:38:03.155173 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:38:03.154896    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:38:03.155173 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:38:03.155043    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:39:07.155029417 +0000 UTC m=+162.651902541 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:38:07.337103 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:38:07.337068    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rzt9z"
Sep 16 16:38:08.118753 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:38:08.118722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:38:34.682705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:38:34.682661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:38:34.683193 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:38:34.682774    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 16:38:34.683193 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:38:34.682841    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:36.68282629 +0000 UTC m=+252.179699391 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : secret "metrics-daemon-secret" not found
Sep 16 16:38:50.305566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:38:50.305510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:39:02.273084 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:39:02.273043    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv" podUID="34007bd9-3504-45a4-9175-3582bffd83d6"
Sep 16 16:39:02.287238 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:39:02.287214    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b5bfg" podUID="d512fb78-3c62-4f8c-af0a-3eb4a888d60d"
Sep 16 16:39:02.533078 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:39:02.533010    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:39:02.533201 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:39:02.533010    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:39:03.984856 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:39:03.984800    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:39:07.206624 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:39:07.206585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:39:07.206624 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:39:07.206621    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:39:07.207036 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:39:07.206733    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 16:39:07.207036 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:39:07.206746    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6788cc6fc8-6n9vv: secret "image-registry-tls" not found
Sep 16 16:39:07.207036 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:39:07.206731    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 16:39:07.207036 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:39:07.206797    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls podName:34007bd9-3504-45a4-9175-3582bffd83d6 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:09.206783595 +0000 UTC m=+284.703656698 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls") pod "image-registry-6788cc6fc8-6n9vv" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6") : secret "image-registry-tls" not found
Sep 16 16:39:07.207036 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:39:07.206810    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls podName:d512fb78-3c62-4f8c-af0a-3eb4a888d60d nodeName:}" failed. No retries permitted until 2026-09-16 16:41:09.20680421 +0000 UTC m=+284.703677311 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls") pod "dns-default-b5bfg" (UID: "d512fb78-3c62-4f8c-af0a-3eb4a888d60d") : secret "dns-default-metrics-tls" not found
Sep 16 16:39:18.971831 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:39:18.971741    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:40:14.069915 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:14.069873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:36.701810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:36.701775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:40:36.702318 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:36.701958    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 16:40:36.702318 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:36.702049    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs podName:07fb4aaa-bae1-4483-8001-e6bbd85f02e4 nodeName:}" failed. No retries permitted until 2026-09-16 16:42:38.702028341 +0000 UTC m=+374.198901458 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs") pod "network-metrics-daemon-9wkss" (UID: "07fb4aaa-bae1-4483-8001-e6bbd85f02e4") : secret "metrics-daemon-secret" not found
Sep 16 16:40:52.432512 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.432439    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"]
Sep 16 16:40:52.434503 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.434483    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.437069 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.437046    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 16 16:40:52.437368 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.437348    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 16 16:40:52.437480 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.437460    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 16 16:40:52.438166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.438137    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:40:52.438166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.438157    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kpf9k\""
Sep 16 16:40:52.439651 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.439572    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-8trxf"]
Sep 16 16:40:52.441952 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.441932    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck"]
Sep 16 16:40:52.442059 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.442043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.443686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.443670    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"]
Sep 16 16:40:52.443814 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.443794    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck"
Sep 16 16:40:52.444451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.444432    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 16 16:40:52.445195 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.445175    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.452372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.452352    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 16 16:40:52.452570 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.452394    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-97mnc\""
Sep 16 16:40:52.452644 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.452618    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:40:52.452644 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.452630    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:40:52.452754 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.452641    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 16:40:52.452754 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.452669    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 16 16:40:52.452884 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.452871    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 16 16:40:52.453015 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.452998    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 16 16:40:52.453516 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.453494    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"]
Sep 16 16:40:52.453918 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.453900    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-lxmdc\""
Sep 16 16:40:52.453990 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.453906    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 16:40:52.455093 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.455073    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cgvvm\""
Sep 16 16:40:52.455182 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.455165    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 16 16:40:52.459684 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.459664    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 16 16:40:52.469160 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.469123    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck"]
Sep 16 16:40:52.469911 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.469857    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-8trxf"]
Sep 16 16:40:52.476333 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.476286    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"]
Sep 16 16:40:52.530008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.529982    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bcr92"]
Sep 16 16:40:52.532015 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.531997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bcr92"
Sep 16 16:40:52.534339 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.534323    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bg2n4\""
Sep 16 16:40:52.537421 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.537403    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"]
Sep 16 16:40:52.539148 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.539130    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-695c688bd8-jc78r"]
Sep 16 16:40:52.539264 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.539249    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.542254 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.542230    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"]
Sep 16 16:40:52.544441 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.543331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.545490 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.545463    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bcr92"]
Sep 16 16:40:52.545600 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.545517    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 16 16:40:52.545600 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.545566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:52.545822 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.545803    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 16 16:40:52.545925 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.545831    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:40:52.546154 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546130    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2sxr9\""
Sep 16 16:40:52.546316 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546299    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 16 16:40:52.546403 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546351    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 16 16:40:52.546592 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546571    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 16 16:40:52.546693 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546577    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 16 16:40:52.546693 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546580    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 16:40:52.546693 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546597    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 16:40:52.546822 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546743    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xssq2\""
Sep 16 16:40:52.546822 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.546753    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 16 16:40:52.547871 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.547855    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:40:52.548306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.548290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 16 16:40:52.548387 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.548325    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p9mjc\""
Sep 16 16:40:52.548387 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.548352    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 16 16:40:52.553603 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.553583    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"]
Sep 16 16:40:52.554360 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.554343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"]
Sep 16 16:40:52.558001 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.557357    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-695c688bd8-jc78r"]
Sep 16 16:40:52.605162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c875s\" (UniqueName: \"kubernetes.io/projected/eced8ef1-c366-44cc-b913-f5148f7c6812-kube-api-access-c875s\") pod \"volume-data-source-validator-86695b7cc7-d2cck\" (UID: \"eced8ef1-c366-44cc-b913-f5148f7c6812\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck"
Sep 16 16:40:52.605257 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8989178-a442-4afa-b2ce-83cfb3677bb6-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.605257 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/911d89a7-998b-43e4-a787-8c3f14775db3-trusted-ca\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.605257 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/911d89a7-998b-43e4-a787-8c3f14775db3-config\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.605372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqn42\" (UniqueName: \"kubernetes.io/projected/911d89a7-998b-43e4-a787-8c3f14775db3-kube-api-access-gqn42\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.605372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfcxc\" (UniqueName: \"kubernetes.io/projected/e8989178-a442-4afa-b2ce-83cfb3677bb6-kube-api-access-tfcxc\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.605372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-std4m\" (UniqueName: \"kubernetes.io/projected/9fbdfae1-d3a8-47d0-971c-a3210c3db180-kube-api-access-std4m\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.605472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fbdfae1-d3a8-47d0-971c-a3210c3db180-serving-cert\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.605472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/911d89a7-998b-43e4-a787-8c3f14775db3-serving-cert\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.605472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605442    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fbdfae1-d3a8-47d0-971c-a3210c3db180-config\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.605472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.605457    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.706287 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzl4b\" (UniqueName: \"kubernetes.io/projected/5d100c30-471d-45a7-ae6e-4c4610cfb308-kube-api-access-jzl4b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.706287 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.706466 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/911d89a7-998b-43e4-a787-8c3f14775db3-trusted-ca\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.706466 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:52.706337    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 16:40:52.706466 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:52.706382    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls podName:e8989178-a442-4afa-b2ce-83cfb3677bb6 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:53.206369269 +0000 UTC m=+268.703242372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b6msc" (UID: "e8989178-a442-4afa-b2ce-83cfb3677bb6") : secret "cluster-monitoring-operator-tls" not found
Sep 16 16:40:52.706466 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/911d89a7-998b-43e4-a787-8c3f14775db3-config\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.706657 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d100c30-471d-45a7-ae6e-4c4610cfb308-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.706657 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.706657 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-default-certificate\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.706796 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fbdfae1-d3a8-47d0-971c-a3210c3db180-config\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.706796 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fbdfae1-d3a8-47d0-971c-a3210c3db180-serving-cert\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.706796 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-stats-auth\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.706796 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpl2v\" (UniqueName: \"kubernetes.io/projected/236b4946-55e8-47aa-b458-50701f89cb1a-kube-api-access-fpl2v\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:52.706796 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706791    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c875s\" (UniqueName: \"kubernetes.io/projected/eced8ef1-c366-44cc-b913-f5148f7c6812-kube-api-access-c875s\") pod \"volume-data-source-validator-86695b7cc7-d2cck\" (UID: \"eced8ef1-c366-44cc-b913-f5148f7c6812\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8989178-a442-4afa-b2ce-83cfb3677bb6-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfcxc\" (UniqueName: \"kubernetes.io/projected/e8989178-a442-4afa-b2ce-83cfb3677bb6-kube-api-access-tfcxc\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pklqc\" (UniqueName: \"kubernetes.io/projected/db442863-9a4f-48cd-826c-2b85bdf9db76-kube-api-access-pklqc\") pod \"network-check-source-6f8b768455-bcr92\" (UID: \"db442863-9a4f-48cd-826c-2b85bdf9db76\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bcr92"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqn42\" (UniqueName: \"kubernetes.io/projected/911d89a7-998b-43e4-a787-8c3f14775db3-kube-api-access-gqn42\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.706996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d100c30-471d-45a7-ae6e-4c4610cfb308-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.707024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-std4m\" (UniqueName: \"kubernetes.io/projected/9fbdfae1-d3a8-47d0-971c-a3210c3db180-kube-api-access-std4m\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.707030 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.707028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/911d89a7-998b-43e4-a787-8c3f14775db3-config\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.707451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.707047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/911d89a7-998b-43e4-a787-8c3f14775db3-serving-cert\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.707451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.707299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7bzr\" (UniqueName: \"kubernetes.io/projected/493536a9-326c-41df-a8bc-cbc55a392894-kube-api-access-v7bzr\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.707451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.707314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/911d89a7-998b-43e4-a787-8c3f14775db3-trusted-ca\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.707451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.707323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fbdfae1-d3a8-47d0-971c-a3210c3db180-config\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.707754 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.707734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8989178-a442-4afa-b2ce-83cfb3677bb6-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.709211 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.709190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fbdfae1-d3a8-47d0-971c-a3210c3db180-serving-cert\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.709306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.709228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/911d89a7-998b-43e4-a787-8c3f14775db3-serving-cert\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.713464 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.713437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:52.715642 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.715615    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfcxc\" (UniqueName: \"kubernetes.io/projected/e8989178-a442-4afa-b2ce-83cfb3677bb6-kube-api-access-tfcxc\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:52.715926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.715906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-std4m\" (UniqueName: \"kubernetes.io/projected/9fbdfae1-d3a8-47d0-971c-a3210c3db180-kube-api-access-std4m\") pod \"service-ca-operator-cf7c776b7-m4494\" (UID: \"9fbdfae1-d3a8-47d0-971c-a3210c3db180\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.716149 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.716135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c875s\" (UniqueName: \"kubernetes.io/projected/eced8ef1-c366-44cc-b913-f5148f7c6812-kube-api-access-c875s\") pod \"volume-data-source-validator-86695b7cc7-d2cck\" (UID: \"eced8ef1-c366-44cc-b913-f5148f7c6812\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck"
Sep 16 16:40:52.716681 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.716665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqn42\" (UniqueName: \"kubernetes.io/projected/911d89a7-998b-43e4-a787-8c3f14775db3-kube-api-access-gqn42\") pod \"console-operator-696884fb9c-8trxf\" (UID: \"911d89a7-998b-43e4-a787-8c3f14775db3\") " pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.745478 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.745458    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"
Sep 16 16:40:52.752891 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.752873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:40:52.759966 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.759786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck"
Sep 16 16:40:52.808971 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.808929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzl4b\" (UniqueName: \"kubernetes.io/projected/5d100c30-471d-45a7-ae6e-4c4610cfb308-kube-api-access-jzl4b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.809096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d100c30-471d-45a7-ae6e-4c4610cfb308-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.809096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.809213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-default-certificate\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.809213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-stats-auth\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.809213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpl2v\" (UniqueName: \"kubernetes.io/projected/236b4946-55e8-47aa-b458-50701f89cb1a-kube-api-access-fpl2v\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:52.809213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.809402 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pklqc\" (UniqueName: \"kubernetes.io/projected/db442863-9a4f-48cd-826c-2b85bdf9db76-kube-api-access-pklqc\") pod \"network-check-source-6f8b768455-bcr92\" (UID: \"db442863-9a4f-48cd-826c-2b85bdf9db76\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bcr92"
Sep 16 16:40:52.809402 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:52.810435 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d100c30-471d-45a7-ae6e-4c4610cfb308-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.810435 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.809637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7bzr\" (UniqueName: \"kubernetes.io/projected/493536a9-326c-41df-a8bc-cbc55a392894-kube-api-access-v7bzr\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.810435 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:52.810062    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 16:40:52.810435 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:52.810122    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls podName:236b4946-55e8-47aa-b458-50701f89cb1a nodeName:}" failed. No retries permitted until 2026-09-16 16:40:53.310103939 +0000 UTC m=+268.806977060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wrp47" (UID: "236b4946-55e8-47aa-b458-50701f89cb1a") : secret "samples-operator-tls" not found
Sep 16 16:40:52.810435 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:52.810339    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:53.310324246 +0000 UTC m=+268.807197361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : configmap references non-existent config key: service-ca.crt
Sep 16 16:40:52.810435 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:52.810408    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 16:40:52.810852 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:52.810447    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:53.310433822 +0000 UTC m=+268.807306924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : secret "router-metrics-certs-default" not found
Sep 16 16:40:52.811167 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.811090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d100c30-471d-45a7-ae6e-4c4610cfb308-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.813125 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.813093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:52.817904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.817635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-stats-auth\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.817904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.817852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d100c30-471d-45a7-ae6e-4c4610cfb308-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.819358 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.819310    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzl4b\" (UniqueName: \"kubernetes.io/projected/5d100c30-471d-45a7-ae6e-4c4610cfb308-kube-api-access-jzl4b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-jxzb7\" (UID: \"5d100c30-471d-45a7-ae6e-4c4610cfb308\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:52.820565 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.819904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-default-certificate\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.820565 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.820083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pklqc\" (UniqueName: \"kubernetes.io/projected/db442863-9a4f-48cd-826c-2b85bdf9db76-kube-api-access-pklqc\") pod \"network-check-source-6f8b768455-bcr92\" (UID: \"db442863-9a4f-48cd-826c-2b85bdf9db76\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bcr92"
Sep 16 16:40:52.820565 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.820510    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpl2v\" (UniqueName: \"kubernetes.io/projected/236b4946-55e8-47aa-b458-50701f89cb1a-kube-api-access-fpl2v\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:52.822032 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.822011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7bzr\" (UniqueName: \"kubernetes.io/projected/493536a9-326c-41df-a8bc-cbc55a392894-kube-api-access-v7bzr\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:52.844390 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.844369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bcr92"
Sep 16 16:40:52.854054 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:52.854038    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"
Sep 16 16:40:53.031136 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.031062    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-8trxf"]
Sep 16 16:40:53.033564 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.033518    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494"]
Sep 16 16:40:53.036141 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:40:53.036120    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911d89a7_998b_43e4_a787_8c3f14775db3.slice/crio-041ac5219f95e6b847809f2138daac667716a2e66aac161b2263b79178e38040 WatchSource:0}: Error finding container 041ac5219f95e6b847809f2138daac667716a2e66aac161b2263b79178e38040: Status 404 returned error can't find the container with id 041ac5219f95e6b847809f2138daac667716a2e66aac161b2263b79178e38040
Sep 16 16:40:53.036603 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:40:53.036583    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fbdfae1_d3a8_47d0_971c_a3210c3db180.slice/crio-925d1553904f0ef05a40df4581c8c80e4c2d924623ddcb5563d48a6c4c8aa9c8 WatchSource:0}: Error finding container 925d1553904f0ef05a40df4581c8c80e4c2d924623ddcb5563d48a6c4c8aa9c8: Status 404 returned error can't find the container with id 925d1553904f0ef05a40df4581c8c80e4c2d924623ddcb5563d48a6c4c8aa9c8
Sep 16 16:40:53.054968 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.054944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck"]
Sep 16 16:40:53.057309 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:40:53.057287    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeced8ef1_c366_44cc_b913_f5148f7c6812.slice/crio-9a60e524889197410bd09a33d4a6455e5ce1c57b600b369c6376f34bed562a15 WatchSource:0}: Error finding container 9a60e524889197410bd09a33d4a6455e5ce1c57b600b369c6376f34bed562a15: Status 404 returned error can't find the container with id 9a60e524889197410bd09a33d4a6455e5ce1c57b600b369c6376f34bed562a15
Sep 16 16:40:53.112375 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.112351    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bcr92"]
Sep 16 16:40:53.114738 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:40:53.114718    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb442863_9a4f_48cd_826c_2b85bdf9db76.slice/crio-e86d12337c7a06a618adaf2c6d855124d9a316fa55069ff18cb7cf73b9cf4de2 WatchSource:0}: Error finding container e86d12337c7a06a618adaf2c6d855124d9a316fa55069ff18cb7cf73b9cf4de2: Status 404 returned error can't find the container with id e86d12337c7a06a618adaf2c6d855124d9a316fa55069ff18cb7cf73b9cf4de2
Sep 16 16:40:53.134889 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.134867    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7"]
Sep 16 16:40:53.139341 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:40:53.139322    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d100c30_471d_45a7_ae6e_4c4610cfb308.slice/crio-4cbad526c3508ea8342108290c84a3ef649a8933d86257e5e49b6977f9e0e9e0 WatchSource:0}: Error finding container 4cbad526c3508ea8342108290c84a3ef649a8933d86257e5e49b6977f9e0e9e0: Status 404 returned error can't find the container with id 4cbad526c3508ea8342108290c84a3ef649a8933d86257e5e49b6977f9e0e9e0
Sep 16 16:40:53.212539 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.212503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:53.212666 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:53.212615    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 16:40:53.212719 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:53.212671    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls podName:e8989178-a442-4afa-b2ce-83cfb3677bb6 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:54.212655761 +0000 UTC m=+269.709528868 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b6msc" (UID: "e8989178-a442-4afa-b2ce-83cfb3677bb6") : secret "cluster-monitoring-operator-tls" not found
Sep 16 16:40:53.313705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.313628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:53.313705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.313667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:53.313930 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:53.313765    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 16:40:53.313930 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:53.313790    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:54.313771246 +0000 UTC m=+269.810644363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : configmap references non-existent config key: service-ca.crt
Sep 16 16:40:53.313930 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.313861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:53.313930 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:53.313925    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls podName:236b4946-55e8-47aa-b458-50701f89cb1a nodeName:}" failed. No retries permitted until 2026-09-16 16:40:54.31390801 +0000 UTC m=+269.810781116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wrp47" (UID: "236b4946-55e8-47aa-b458-50701f89cb1a") : secret "samples-operator-tls" not found
Sep 16 16:40:53.314147 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:53.313975    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 16:40:53.314147 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:53.314001    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:54.313995063 +0000 UTC m=+269.810868165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : secret "router-metrics-certs-default" not found
Sep 16 16:40:53.792917 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.792879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7" event={"ID":"5d100c30-471d-45a7-ae6e-4c4610cfb308","Type":"ContainerStarted","Data":"4cbad526c3508ea8342108290c84a3ef649a8933d86257e5e49b6977f9e0e9e0"}
Sep 16 16:40:53.795562 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.795467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bcr92" event={"ID":"db442863-9a4f-48cd-826c-2b85bdf9db76","Type":"ContainerStarted","Data":"401d532c295b016d84f2476fc6f04a05196e2a06d9873b25bb32f04b4f733a31"}
Sep 16 16:40:53.795681 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.795520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bcr92" event={"ID":"db442863-9a4f-48cd-826c-2b85bdf9db76","Type":"ContainerStarted","Data":"e86d12337c7a06a618adaf2c6d855124d9a316fa55069ff18cb7cf73b9cf4de2"}
Sep 16 16:40:53.797797 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.797740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck" event={"ID":"eced8ef1-c366-44cc-b913-f5148f7c6812","Type":"ContainerStarted","Data":"9a60e524889197410bd09a33d4a6455e5ce1c57b600b369c6376f34bed562a15"}
Sep 16 16:40:53.799296 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.799263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494" event={"ID":"9fbdfae1-d3a8-47d0-971c-a3210c3db180","Type":"ContainerStarted","Data":"925d1553904f0ef05a40df4581c8c80e4c2d924623ddcb5563d48a6c4c8aa9c8"}
Sep 16 16:40:53.801129 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.801084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" event={"ID":"911d89a7-998b-43e4-a787-8c3f14775db3","Type":"ContainerStarted","Data":"041ac5219f95e6b847809f2138daac667716a2e66aac161b2263b79178e38040"}
Sep 16 16:40:53.813425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:53.813381    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bcr92" podStartSLOduration=1.813368104 podStartE2EDuration="1.813368104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:40:53.811561678 +0000 UTC m=+269.308434802" watchObservedRunningTime="2026-09-16 16:40:53.813368104 +0000 UTC m=+269.310241228"
Sep 16 16:40:54.220455 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:54.220318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:54.220634 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:54.220499    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 16:40:54.220634 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:54.220578    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls podName:e8989178-a442-4afa-b2ce-83cfb3677bb6 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:56.220560414 +0000 UTC m=+271.717433530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b6msc" (UID: "e8989178-a442-4afa-b2ce-83cfb3677bb6") : secret "cluster-monitoring-operator-tls" not found
Sep 16 16:40:54.321860 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:54.321441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:54.321860 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:54.321541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:54.321860 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:54.321581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:54.321860 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:54.321681    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 16:40:54.321860 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:54.321738    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 16:40:54.321860 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:54.321743    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:56.321725047 +0000 UTC m=+271.818598168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : secret "router-metrics-certs-default" not found
Sep 16 16:40:54.321860 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:54.321772    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls podName:236b4946-55e8-47aa-b458-50701f89cb1a nodeName:}" failed. No retries permitted until 2026-09-16 16:40:56.321762152 +0000 UTC m=+271.818635258 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wrp47" (UID: "236b4946-55e8-47aa-b458-50701f89cb1a") : secret "samples-operator-tls" not found
Sep 16 16:40:54.321860 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:54.321832    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:40:56.321822503 +0000 UTC m=+271.818695609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : configmap references non-existent config key: service-ca.crt
Sep 16 16:40:54.806724 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:54.806637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck" event={"ID":"eced8ef1-c366-44cc-b913-f5148f7c6812","Type":"ContainerStarted","Data":"22b88357b7128beced9f76160f63941e93b41311224764ed7aef494e0ca3b305"}
Sep 16 16:40:54.824372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:54.824328    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-d2cck" podStartSLOduration=1.344308638 podStartE2EDuration="2.824309218s" totalImagesPullingTime="1.48000058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:40:54.821856672 +0000 UTC m=+270.318729797" watchObservedRunningTime="2026-09-16 16:40:54.824309218 +0000 UTC m=+270.321182344"
Sep 16 16:40:54.909885 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:54.906004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:55.005435 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:55.005397    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:56.029208 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.029170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:56.035873 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.035840    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lb6ps"]
Sep 16 16:40:56.038644 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.038621    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.041851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.041824    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 16:40:56.041965 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.041865    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 16:40:56.041965 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.041917    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nnmt\""
Sep 16 16:40:56.042704 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.042680    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 16:40:56.042828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.042691    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 16:40:56.050639 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.050616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lb6ps"]
Sep 16 16:40:56.103997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.103963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:56.140002 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.139942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a0fcb09c-502d-4494-99fc-a128ca504f2e-crio-socket\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.140183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.140050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0fcb09c-502d-4494-99fc-a128ca504f2e-data-volume\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.140183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.140133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.140306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.140259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzz6x\" (UniqueName: \"kubernetes.io/projected/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-api-access-fzz6x\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.140359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.140327    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.241358 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.241274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0fcb09c-502d-4494-99fc-a128ca504f2e-data-volume\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.241358 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.241321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:40:56.241486 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.241359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.241486 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.241422    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 16:40:56.241608 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.241499    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:56.241608 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.241424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzz6x\" (UniqueName: \"kubernetes.io/projected/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-api-access-fzz6x\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.241608 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.241502    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls podName:e8989178-a442-4afa-b2ce-83cfb3677bb6 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:00.241482186 +0000 UTC m=+275.738355308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b6msc" (UID: "e8989178-a442-4afa-b2ce-83cfb3677bb6") : secret "cluster-monitoring-operator-tls" not found
Sep 16 16:40:56.241608 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.241578    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:40:56.741565201 +0000 UTC m=+272.238438318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:56.241784 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.241638    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0fcb09c-502d-4494-99fc-a128ca504f2e-data-volume\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.241784 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.241638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.241784 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.241721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a0fcb09c-502d-4494-99fc-a128ca504f2e-crio-socket\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.241784 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.241757    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:40:56.241896 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.241798    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:40:56.741785217 +0000 UTC m=+272.238658337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : configmap "kube-rbac-proxy" not found
Sep 16 16:40:56.241896 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.241829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a0fcb09c-502d-4494-99fc-a128ca504f2e-crio-socket\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.250287 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.250265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzz6x\" (UniqueName: \"kubernetes.io/projected/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-api-access-fzz6x\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.342985 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.342948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:56.343146 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.343005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:40:56.343146 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.343038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:40:56.343146 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.343096    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 16:40:56.343248 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.343152    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:00.343137765 +0000 UTC m=+275.840010868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : secret "router-metrics-certs-default" not found
Sep 16 16:40:56.343248 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.343191    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 16:40:56.343248 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.343241    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls podName:236b4946-55e8-47aa-b458-50701f89cb1a nodeName:}" failed. No retries permitted until 2026-09-16 16:41:00.343226164 +0000 UTC m=+275.840099280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wrp47" (UID: "236b4946-55e8-47aa-b458-50701f89cb1a") : secret "samples-operator-tls" not found
Sep 16 16:40:56.343348 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.343310    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:00.343299672 +0000 UTC m=+275.840172797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : configmap references non-existent config key: service-ca.crt
Sep 16 16:40:56.746613 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.746579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.746778 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.746743    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:56.746854 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.746768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:56.746854 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.746813    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:40:57.746793834 +0000 UTC m=+273.243666959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:56.746950 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.746870    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:40:56.746950 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:56.746922    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:40:57.746905719 +0000 UTC m=+273.243778835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : configmap "kube-rbac-proxy" not found
Sep 16 16:40:56.815058 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.814944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7" event={"ID":"5d100c30-471d-45a7-ae6e-4c4610cfb308","Type":"ContainerStarted","Data":"12757acbc8c9e9808da18e575694210c4043c70814005ca4bf2d4f05654d516d"}
Sep 16 16:40:56.816594 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.816566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494" event={"ID":"9fbdfae1-d3a8-47d0-971c-a3210c3db180","Type":"ContainerStarted","Data":"2b58431c9de0aacf541aacf51cdc633f8ca50187bd10d62c8e16d71d44cefd75"}
Sep 16 16:40:56.818265 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.818240    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/0.log"
Sep 16 16:40:56.818418 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.818280    2577 generic.go:356] "Generic (PLEG): container finished" podID="911d89a7-998b-43e4-a787-8c3f14775db3" containerID="f0a8160e0f55187590ffce7b75cf7d3edf97d62cffdfc52b99f4e4df51be8a30" exitCode=255
Sep 16 16:40:56.818418 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.818316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" event={"ID":"911d89a7-998b-43e4-a787-8c3f14775db3","Type":"ContainerDied","Data":"f0a8160e0f55187590ffce7b75cf7d3edf97d62cffdfc52b99f4e4df51be8a30"}
Sep 16 16:40:56.818583 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.818565    2577 scope.go:117] "RemoveContainer" containerID="f0a8160e0f55187590ffce7b75cf7d3edf97d62cffdfc52b99f4e4df51be8a30"
Sep 16 16:40:56.830355 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.830296    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7" podStartSLOduration=1.713452653 podStartE2EDuration="4.830279068s" totalImagesPullingTime="3.116826415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:40:56.830010022 +0000 UTC m=+272.326883147" watchObservedRunningTime="2026-09-16 16:40:56.830279068 +0000 UTC m=+272.327152187"
Sep 16 16:40:56.846270 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:56.846214    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494" podStartSLOduration=1.6338211519999999 podStartE2EDuration="4.846194032s" totalImagesPullingTime="3.21237288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:40:56.84518231 +0000 UTC m=+272.342055435" watchObservedRunningTime="2026-09-16 16:40:56.846194032 +0000 UTC m=+272.343067157"
Sep 16 16:40:57.178994 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.178954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:57.249803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.249770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:57.420818 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.420783    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:57.540406 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.540309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:57.639801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.639761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:57.704259 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.704222    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:57.756658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.756633    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:57.756795 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.756702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:57.756795 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:57.756761    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:40:57.756865 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:57.756800    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:57.756865 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:57.756838    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:40:59.756817328 +0000 UTC m=+275.253690435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : configmap "kube-rbac-proxy" not found
Sep 16 16:40:57.756865 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:57.756858    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:40:59.756846295 +0000 UTC m=+275.253719403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:57.823163 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.823100    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/1.log"
Sep 16 16:40:57.823487 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.823468    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/0.log"
Sep 16 16:40:57.823595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.823509    2577 generic.go:356] "Generic (PLEG): container finished" podID="911d89a7-998b-43e4-a787-8c3f14775db3" containerID="982ec8ecc12783dc0fae2921cf21bf80a625fd5bc02dc9d0f4e21d789034c3fd" exitCode=255
Sep 16 16:40:57.823658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.823597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" event={"ID":"911d89a7-998b-43e4-a787-8c3f14775db3","Type":"ContainerDied","Data":"982ec8ecc12783dc0fae2921cf21bf80a625fd5bc02dc9d0f4e21d789034c3fd"}
Sep 16 16:40:57.823658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.823642    2577 scope.go:117] "RemoveContainer" containerID="f0a8160e0f55187590ffce7b75cf7d3edf97d62cffdfc52b99f4e4df51be8a30"
Sep 16 16:40:57.823846 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.823821    2577 scope.go:117] "RemoveContainer" containerID="982ec8ecc12783dc0fae2921cf21bf80a625fd5bc02dc9d0f4e21d789034c3fd"
Sep 16 16:40:57.824030 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:57.824006    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-8trxf_openshift-console-operator(911d89a7-998b-43e4-a787-8c3f14775db3)\"" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podUID="911d89a7-998b-43e4-a787-8c3f14775db3"
Sep 16 16:40:57.893281 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.893251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:57.978939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.978917    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-htltb_83be222b-05e2-44b0-90c5-bf387e175d3e/dns-node-resolver/0.log"
Sep 16 16:40:57.995538 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:57.995509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:58.778378 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:58.778350    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ft9d_b1d7e520-75a5-4587-b4f3-77ecdf1b2460/node-ca/0.log"
Sep 16 16:40:58.828718 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:58.828696    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/1.log"
Sep 16 16:40:58.829006 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:58.828993    2577 scope.go:117] "RemoveContainer" containerID="982ec8ecc12783dc0fae2921cf21bf80a625fd5bc02dc9d0f4e21d789034c3fd"
Sep 16 16:40:58.829165 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:58.829148    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-8trxf_openshift-console-operator(911d89a7-998b-43e4-a787-8c3f14775db3)\"" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podUID="911d89a7-998b-43e4-a787-8c3f14775db3"
Sep 16 16:40:59.568162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:59.568128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:59.668847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:59.668812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:40:59.774050 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:59.774023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:59.774194 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:59.774095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:40:59.774194 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:59.774137    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:40:59.774283 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:59.774195    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:41:03.774180506 +0000 UTC m=+279.271053607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : configmap "kube-rbac-proxy" not found
Sep 16 16:40:59.774283 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:59.774209    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:59.774283 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:40:59.774257    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:41:03.774245693 +0000 UTC m=+279.271118794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:40:59.793557 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:40:59.793521    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rds89_4423a1a4-565b-402b-a813-b228a1d2213c/serve-healthcheck-canary/0.log"
Sep 16 16:41:00.260895 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:00.260858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:00.278059 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:00.278040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:41:00.278166 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:00.278153    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 16:41:00.278216 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:00.278206    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls podName:e8989178-a442-4afa-b2ce-83cfb3677bb6 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:08.278192355 +0000 UTC m=+283.775065456 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b6msc" (UID: "e8989178-a442-4afa-b2ce-83cfb3677bb6") : secret "cluster-monitoring-operator-tls" not found
Sep 16 16:41:00.359783 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:00.359755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:00.378987 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:00.378963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:00.379047 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:00.378999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:41:00.379100 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:00.379090    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 16 16:41:00.379131 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:00.379112    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:08.379093865 +0000 UTC m=+283.875966985 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : configmap references non-existent config key: service-ca.crt
Sep 16 16:41:00.379193 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:00.379178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:00.379193 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:00.379189    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls podName:236b4946-55e8-47aa-b458-50701f89cb1a nodeName:}" failed. No retries permitted until 2026-09-16 16:41:08.379175656 +0000 UTC m=+283.876048771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-wrp47" (UID: "236b4946-55e8-47aa-b458-50701f89cb1a") : secret "samples-operator-tls" not found
Sep 16 16:41:00.379261 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:00.379252    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 16:41:00.379296 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:00.379289    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:08.379278073 +0000 UTC m=+283.876151185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : secret "router-metrics-certs-default" not found
Sep 16 16:41:00.660667 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:00.660636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:02.753893 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:02.753815    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:41:02.753893 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:02.753874    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:41:02.754243 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:02.754166    2577 scope.go:117] "RemoveContainer" containerID="982ec8ecc12783dc0fae2921cf21bf80a625fd5bc02dc9d0f4e21d789034c3fd"
Sep 16 16:41:02.755095 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:02.755073    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-8trxf_openshift-console-operator(911d89a7-998b-43e4-a787-8c3f14775db3)\"" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podUID="911d89a7-998b-43e4-a787-8c3f14775db3"
Sep 16 16:41:03.128404 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:03.128316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:03.634102 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:03.634064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:03.745493 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:03.745455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:03.805852 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:03.805825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:41:03.806206 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:03.805895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:41:03.806206 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:03.805960    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:41:03.806206 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:03.806014    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:41:11.806000334 +0000 UTC m=+287.302873437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:41:03.806206 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:03.805963    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 16:41:03.806206 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:03.806047    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:41:11.806039677 +0000 UTC m=+287.302912778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : configmap "kube-rbac-proxy" not found
Sep 16 16:41:03.857318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:03.857285    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:03.947119 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:03.947096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:04.048072 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:04.048044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:04.509400 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:04.509356    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:05.110896 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:05.110859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:05.505349 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:05.505313    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:05.533776 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:05.533752    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-b5bfg" podUID="d512fb78-3c62-4f8c-af0a-3eb4a888d60d"
Sep 16 16:41:05.533862 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:05.533755    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv" podUID="34007bd9-3504-45a4-9175-3582bffd83d6"
Sep 16 16:41:05.706373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:05.706342    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:05.848559 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:05.848468    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:41:05.848686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:05.848469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:41:05.907200 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:05.907173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:06.523387 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:06.523355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:06.908358 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:06.908327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:07.124365 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:07.124326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:07.305733 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:07.305659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:07.502523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:07.502493    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:07.743459 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:07.743423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:07.842284 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:07.842235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:07.942149 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:07.942119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:08.344916 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.344885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:41:08.345074 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:08.345034    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 16 16:41:08.345116 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:08.345108    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls podName:e8989178-a442-4afa-b2ce-83cfb3677bb6 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:24.345089485 +0000 UTC m=+299.841962599 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-b6msc" (UID: "e8989178-a442-4afa-b2ce-83cfb3677bb6") : secret "cluster-monitoring-operator-tls" not found
Sep 16 16:41:08.446195 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.446162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:08.446330 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.446226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:08.446330 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.446259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:41:08.446421 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:08.446402    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle podName:493536a9-326c-41df-a8bc-cbc55a392894 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:24.446386308 +0000 UTC m=+299.943259409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle") pod "router-default-695c688bd8-jc78r" (UID: "493536a9-326c-41df-a8bc-cbc55a392894") : configmap references non-existent config key: service-ca.crt
Sep 16 16:41:08.448650 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.448618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/236b4946-55e8-47aa-b458-50701f89cb1a-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-wrp47\" (UID: \"236b4946-55e8-47aa-b458-50701f89cb1a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:41:08.448752 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.448729    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/493536a9-326c-41df-a8bc-cbc55a392894-metrics-certs\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:08.466154 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.466133    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"
Sep 16 16:41:08.735001 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.734911    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47"]
Sep 16 16:41:08.858099 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.858069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47" event={"ID":"236b4946-55e8-47aa-b458-50701f89cb1a","Type":"ContainerStarted","Data":"e46993b18b3a50b4f0c7f6123a49cbd0d7954241dc65178eeadb5c1f603a9506"}
Sep 16 16:41:08.908332 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:08.908309    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:09.234484 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.234448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:09.251718 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.251690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:41:09.251825 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.251722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:41:09.253987 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.253964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d512fb78-3c62-4f8c-af0a-3eb4a888d60d-metrics-tls\") pod \"dns-default-b5bfg\" (UID: \"d512fb78-3c62-4f8c-af0a-3eb4a888d60d\") " pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:41:09.254122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.254103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"image-registry-6788cc6fc8-6n9vv\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") " pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:41:09.335293 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.335262    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:09.452210 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.452184    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sm9dz\""
Sep 16 16:41:09.452343 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.452318    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bqrg9\""
Sep 16 16:41:09.460299 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.460283    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:41:09.460394 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.460375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:41:09.560452 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.560420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:09.707919 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.707885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:09.736517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.736433    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b5bfg"]
Sep 16 16:41:09.739081 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:09.739057    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd512fb78_3c62_4f8c_af0a_3eb4a888d60d.slice/crio-d814164f5631f02bb6be4dc34241385c1e277f123ab7ab227933becbc2cb234c WatchSource:0}: Error finding container d814164f5631f02bb6be4dc34241385c1e277f123ab7ab227933becbc2cb234c: Status 404 returned error can't find the container with id d814164f5631f02bb6be4dc34241385c1e277f123ab7ab227933becbc2cb234c
Sep 16 16:41:09.743508 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.743265    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6788cc6fc8-6n9vv"]
Sep 16 16:41:09.745986 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:09.745965    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34007bd9_3504_45a4_9175_3582bffd83d6.slice/crio-2daf848cbaf6510b0b0296c3d36dddcc96aff22167659dbf3c7577f111d6db0b WatchSource:0}: Error finding container 2daf848cbaf6510b0b0296c3d36dddcc96aff22167659dbf3c7577f111d6db0b: Status 404 returned error can't find the container with id 2daf848cbaf6510b0b0296c3d36dddcc96aff22167659dbf3c7577f111d6db0b
Sep 16 16:41:09.866321 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.866285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5bfg" event={"ID":"d512fb78-3c62-4f8c-af0a-3eb4a888d60d","Type":"ContainerStarted","Data":"d814164f5631f02bb6be4dc34241385c1e277f123ab7ab227933becbc2cb234c"}
Sep 16 16:41:09.868078 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.868056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv" event={"ID":"34007bd9-3504-45a4-9175-3582bffd83d6","Type":"ContainerStarted","Data":"8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610"}
Sep 16 16:41:09.868200 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.868091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv" event={"ID":"34007bd9-3504-45a4-9175-3582bffd83d6","Type":"ContainerStarted","Data":"2daf848cbaf6510b0b0296c3d36dddcc96aff22167659dbf3c7577f111d6db0b"}
Sep 16 16:41:09.868200 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.868195    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:41:09.887754 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:09.887704    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv" podStartSLOduration=267.887684164 podStartE2EDuration="4m27.887684164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:09.887479409 +0000 UTC m=+285.384352536" watchObservedRunningTime="2026-09-16 16:41:09.887684164 +0000 UTC m=+285.384557290"
Sep 16 16:41:10.015658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:10.015600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:10.118723 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:10.118632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:10.702787 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:10.702743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:10.874599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:10.874567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47" event={"ID":"236b4946-55e8-47aa-b458-50701f89cb1a","Type":"ContainerStarted","Data":"7b7b384d9f7ab03c6f35f8d265fa2146b3b14835c79707f7fd0eea419f274d73"}
Sep 16 16:41:11.105335 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.105245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:11.511227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.511159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:11.707917 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.707868    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:11.872292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.872266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:41:11.872394 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.872374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:41:11.872792 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:11.872493    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 16:41:11.872792 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:11.872568    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls podName:a0fcb09c-502d-4494-99fc-a128ca504f2e nodeName:}" failed. No retries permitted until 2026-09-16 16:41:27.872549509 +0000 UTC m=+303.369422625 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lb6ps" (UID: "a0fcb09c-502d-4494-99fc-a128ca504f2e") : secret "insights-runtime-extractor-tls" not found
Sep 16 16:41:11.873092 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.872855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a0fcb09c-502d-4494-99fc-a128ca504f2e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:41:11.882134 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.882100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47" event={"ID":"236b4946-55e8-47aa-b458-50701f89cb1a","Type":"ContainerStarted","Data":"ecc1a53573f0c5b148f8fe8eb96637cf247bd11c0862b0439d10e833cd628b88"}
Sep 16 16:41:11.883859 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.883835    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5bfg" event={"ID":"d512fb78-3c62-4f8c-af0a-3eb4a888d60d","Type":"ContainerStarted","Data":"6f8ea96e34c5f50a8dc9a44ebf784a8df60bc654562bc08ece4bb6ed7ea0d426"}
Sep 16 16:41:11.902078 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.902028    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-wrp47" podStartSLOduration=17.932903361 podStartE2EDuration="19.902010925s" totalImagesPullingTime="1.969107564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:11.899951992 +0000 UTC m=+287.396825131" watchObservedRunningTime="2026-09-16 16:41:11.902010925 +0000 UTC m=+287.398884050"
Sep 16 16:41:11.902214 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:11.902111    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:12.313602 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:12.313562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:12.889083 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:12.889043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b5bfg" event={"ID":"d512fb78-3c62-4f8c-af0a-3eb4a888d60d","Type":"ContainerStarted","Data":"15fff48828cda1275a184c25650747685231db5ce2d8a41d4ddc4ca59839e119"}
Sep 16 16:41:12.909096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:12.909045    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b5bfg" podStartSLOduration=251.874950555 podStartE2EDuration="4m13.909029834s" totalImagesPullingTime="2.034079279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:12.908957343 +0000 UTC m=+288.405830469" watchObservedRunningTime="2026-09-16 16:41:12.909029834 +0000 UTC m=+288.405902949"
Sep 16 16:41:12.992767 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:12.992745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:13.104566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:13.104514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:13.302625 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:13.302524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:13.504336 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:13.504302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:13.892259 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:13.892235    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:41:14.099689 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:14.099639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:14.305783 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:14.305686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:15.971235 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:15.971203    2577 scope.go:117] "RemoveContainer" containerID="982ec8ecc12783dc0fae2921cf21bf80a625fd5bc02dc9d0f4e21d789034c3fd"
Sep 16 16:41:16.105470 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:16.105440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:16.204682 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:16.204648    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:16.905047 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:16.905023    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:41:16.905416 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:16.905400    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/1.log"
Sep 16 16:41:16.905457 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:16.905435    2577 generic.go:356] "Generic (PLEG): container finished" podID="911d89a7-998b-43e4-a787-8c3f14775db3" containerID="b749b8d59d1e12e339a36bd1c7926434bb191f805166575f56f2a836357b2f21" exitCode=255
Sep 16 16:41:16.905499 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:16.905485    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" event={"ID":"911d89a7-998b-43e4-a787-8c3f14775db3","Type":"ContainerDied","Data":"b749b8d59d1e12e339a36bd1c7926434bb191f805166575f56f2a836357b2f21"}
Sep 16 16:41:16.905568 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:16.905552    2577 scope.go:117] "RemoveContainer" containerID="982ec8ecc12783dc0fae2921cf21bf80a625fd5bc02dc9d0f4e21d789034c3fd"
Sep 16 16:41:16.905858 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:16.905838    2577 scope.go:117] "RemoveContainer" containerID="b749b8d59d1e12e339a36bd1c7926434bb191f805166575f56f2a836357b2f21"
Sep 16 16:41:16.906017 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:16.906000    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-8trxf_openshift-console-operator(911d89a7-998b-43e4-a787-8c3f14775db3)\"" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podUID="911d89a7-998b-43e4-a787-8c3f14775db3"
Sep 16 16:41:17.909688 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:17.909662    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:41:21.220958 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.220925    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6788cc6fc8-6n9vv"]
Sep 16 16:41:21.258069 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.258040    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67d8774cb9-gwd48"]
Sep 16 16:41:21.263888 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.263855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.274487 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.274458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67d8774cb9-gwd48"]
Sep 16 16:41:21.342116 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.342086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kz5g\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-kube-api-access-5kz5g\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.342276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.342121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2d0d8200-94cf-41bb-8985-9cb2d04dea16-image-registry-private-configuration\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.342276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.342191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-bound-sa-token\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.342276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.342241    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2d0d8200-94cf-41bb-8985-9cb2d04dea16-registry-certificates\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.342392 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.342280    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2d0d8200-94cf-41bb-8985-9cb2d04dea16-installation-pull-secrets\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.342392 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.342336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-registry-tls\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.342392 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.342360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2d0d8200-94cf-41bb-8985-9cb2d04dea16-ca-trust-extracted\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.342392 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.342378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d0d8200-94cf-41bb-8985-9cb2d04dea16-trusted-ca\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.380579 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.380549    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:21.442830 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.442802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-registry-tls\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.442830 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.442829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2d0d8200-94cf-41bb-8985-9cb2d04dea16-ca-trust-extracted\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.442997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.442845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d0d8200-94cf-41bb-8985-9cb2d04dea16-trusted-ca\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.442997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.442869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kz5g\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-kube-api-access-5kz5g\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.442997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.442901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2d0d8200-94cf-41bb-8985-9cb2d04dea16-image-registry-private-configuration\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.442997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.442952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-bound-sa-token\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.442997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.442974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2d0d8200-94cf-41bb-8985-9cb2d04dea16-registry-certificates\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.443218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.443001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2d0d8200-94cf-41bb-8985-9cb2d04dea16-installation-pull-secrets\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.443283 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.443260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2d0d8200-94cf-41bb-8985-9cb2d04dea16-ca-trust-extracted\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.444049 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.444023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2d0d8200-94cf-41bb-8985-9cb2d04dea16-registry-certificates\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.444209 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.444188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2d0d8200-94cf-41bb-8985-9cb2d04dea16-trusted-ca\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.445582 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.445560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-registry-tls\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.445641 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.445591    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2d0d8200-94cf-41bb-8985-9cb2d04dea16-image-registry-private-configuration\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.445641 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.445608    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2d0d8200-94cf-41bb-8985-9cb2d04dea16-installation-pull-secrets\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.451027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.451005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-bound-sa-token\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.451126 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.451013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kz5g\" (UniqueName: \"kubernetes.io/projected/2d0d8200-94cf-41bb-8985-9cb2d04dea16-kube-api-access-5kz5g\") pod \"image-registry-67d8774cb9-gwd48\" (UID: \"2d0d8200-94cf-41bb-8985-9cb2d04dea16\") " pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.572576 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.572495    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.845576 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.845483    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67d8774cb9-gwd48"]
Sep 16 16:41:21.848330 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:21.848307    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d0d8200_94cf_41bb_8985_9cb2d04dea16.slice/crio-eecf17e37fa727869c00a204f71349e75a0221c760e658e3779231cfadae61dc WatchSource:0}: Error finding container eecf17e37fa727869c00a204f71349e75a0221c760e658e3779231cfadae61dc: Status 404 returned error can't find the container with id eecf17e37fa727869c00a204f71349e75a0221c760e658e3779231cfadae61dc
Sep 16 16:41:21.923475 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.923436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67d8774cb9-gwd48" event={"ID":"2d0d8200-94cf-41bb-8985-9cb2d04dea16","Type":"ContainerStarted","Data":"f77e400186ecba71313e3f151d86223d421524667bab1f39d1a5ad54fbf830b9"}
Sep 16 16:41:21.923589 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.923497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67d8774cb9-gwd48" event={"ID":"2d0d8200-94cf-41bb-8985-9cb2d04dea16","Type":"ContainerStarted","Data":"eecf17e37fa727869c00a204f71349e75a0221c760e658e3779231cfadae61dc"}
Sep 16 16:41:21.923589 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.923569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:21.942913 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:21.942861    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67d8774cb9-gwd48" podStartSLOduration=0.94284011 podStartE2EDuration="942.84011ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:21.941695778 +0000 UTC m=+297.438568903" watchObservedRunningTime="2026-09-16 16:41:21.94284011 +0000 UTC m=+297.439713235"
Sep 16 16:41:21.972619 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:21.972587    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9wkss" podUID="07fb4aaa-bae1-4483-8001-e6bbd85f02e4"
Sep 16 16:41:22.752978 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:22.752951    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:41:22.753475 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:22.753211    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:41:22.753475 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:22.753322    2577 scope.go:117] "RemoveContainer" containerID="b749b8d59d1e12e339a36bd1c7926434bb191f805166575f56f2a836357b2f21"
Sep 16 16:41:22.753617 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:22.753508    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-8trxf_openshift-console-operator(911d89a7-998b-43e4-a787-8c3f14775db3)\"" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podUID="911d89a7-998b-43e4-a787-8c3f14775db3"
Sep 16 16:41:22.926425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:22.926395    2577 scope.go:117] "RemoveContainer" containerID="b749b8d59d1e12e339a36bd1c7926434bb191f805166575f56f2a836357b2f21"
Sep 16 16:41:22.926579 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:22.926559    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-8trxf_openshift-console-operator(911d89a7-998b-43e4-a787-8c3f14775db3)\"" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podUID="911d89a7-998b-43e4-a787-8c3f14775db3"
Sep 16 16:41:23.770783 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:23.770752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:23.897589 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:23.897561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b5bfg"
Sep 16 16:41:24.369009 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.368967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:41:24.371434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.371409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8989178-a442-4afa-b2ce-83cfb3677bb6-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-b6msc\" (UID: \"e8989178-a442-4afa-b2ce-83cfb3677bb6\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:41:24.469695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.469656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:24.470289 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.470264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/493536a9-326c-41df-a8bc-cbc55a392894-service-ca-bundle\") pod \"router-default-695c688bd8-jc78r\" (UID: \"493536a9-326c-41df-a8bc-cbc55a392894\") " pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:24.564466 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.564438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"
Sep 16 16:41:24.660930 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.660897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:24.844205 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.844182    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:41:24.844583 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.844292    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:41:24.854082 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.854063    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 16:41:24.880623 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.880544    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc"]
Sep 16 16:41:24.882776 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:24.882756    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8989178_a442_4afa_b2ce_83cfb3677bb6.slice/crio-972493d9e5b937eb4d8136cbd89dfc3fe9d348295f9c49b4fa6a50550fa2b3dd WatchSource:0}: Error finding container 972493d9e5b937eb4d8136cbd89dfc3fe9d348295f9c49b4fa6a50550fa2b3dd: Status 404 returned error can't find the container with id 972493d9e5b937eb4d8136cbd89dfc3fe9d348295f9c49b4fa6a50550fa2b3dd
Sep 16 16:41:25.096775 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.935247    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc" event={"ID":"e8989178-a442-4afa-b2ce-83cfb3677bb6","Type":"ContainerStarted","Data":"972493d9e5b937eb4d8136cbd89dfc3fe9d348295f9c49b4fa6a50550fa2b3dd"}
Sep 16 16:41:25.096775 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:24.949854    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-695c688bd8-jc78r"]
Sep 16 16:41:25.096775 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:24.951855    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493536a9_326c_41df_a8bc_cbc55a392894.slice/crio-609cf7f1711cf769afbb0a1386c6569929831e1661f5e07d75ebab86cb56b8d0 WatchSource:0}: Error finding container 609cf7f1711cf769afbb0a1386c6569929831e1661f5e07d75ebab86cb56b8d0: Status 404 returned error can't find the container with id 609cf7f1711cf769afbb0a1386c6569929831e1661f5e07d75ebab86cb56b8d0
Sep 16 16:41:25.940502 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:25.940460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-695c688bd8-jc78r" event={"ID":"493536a9-326c-41df-a8bc-cbc55a392894","Type":"ContainerStarted","Data":"f79fad65db3c344ba430e8bbd853fe3e25125b5228378db338f2860be6c4a5fb"}
Sep 16 16:41:25.940939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:25.940578    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-695c688bd8-jc78r" event={"ID":"493536a9-326c-41df-a8bc-cbc55a392894","Type":"ContainerStarted","Data":"609cf7f1711cf769afbb0a1386c6569929831e1661f5e07d75ebab86cb56b8d0"}
Sep 16 16:41:25.961808 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:25.961751    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-695c688bd8-jc78r" podStartSLOduration=33.961731183 podStartE2EDuration="33.961731183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:25.960009223 +0000 UTC m=+301.456882350" watchObservedRunningTime="2026-09-16 16:41:25.961731183 +0000 UTC m=+301.458604300"
Sep 16 16:41:26.661886 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:26.661849    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:26.664937 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:26.664908    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:26.943886 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:26.943793    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:26.944910 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:26.944885    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-695c688bd8-jc78r"
Sep 16 16:41:27.383981 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:27.383905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:27.484232 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:27.484193    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:27.796569 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:27.796489    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:27.898117 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:27.898080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:41:27.900370 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:27.900348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a0fcb09c-502d-4494-99fc-a128ca504f2e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lb6ps\" (UID: \"a0fcb09c-502d-4494-99fc-a128ca504f2e\") " pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:41:27.951879 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:27.951844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc" event={"ID":"e8989178-a442-4afa-b2ce-83cfb3677bb6","Type":"ContainerStarted","Data":"e8743d440ae024d872fb2e5c3544f62588dc4f1624fbb553a7a9acc19461c20f"}
Sep 16 16:41:27.969024 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:27.968987    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-b6msc" podStartSLOduration=33.78345099 podStartE2EDuration="35.968972261s" totalImagesPullingTime="2.185521271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:27.968744677 +0000 UTC m=+303.465617802" watchObservedRunningTime="2026-09-16 16:41:27.968972261 +0000 UTC m=+303.465845386"
Sep 16 16:41:28.150893 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.150864    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8nnmt\""
Sep 16 16:41:28.158747 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.158730    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lb6ps"
Sep 16 16:41:28.434293 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.434222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lb6ps"]
Sep 16 16:41:28.436991 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:28.436966    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0fcb09c_502d_4494_99fc_a128ca504f2e.slice/crio-edfdca08804c6cc477a3b4d2bd756065ab9705d83db9ff589a83eb739c517555 WatchSource:0}: Error finding container edfdca08804c6cc477a3b4d2bd756065ab9705d83db9ff589a83eb739c517555: Status 404 returned error can't find the container with id edfdca08804c6cc477a3b4d2bd756065ab9705d83db9ff589a83eb739c517555
Sep 16 16:41:28.488554 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.488520    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:41:28.508627 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.508597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:28.607926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.607894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:28.918191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.918164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:28.956157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.956129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lb6ps" event={"ID":"a0fcb09c-502d-4494-99fc-a128ca504f2e","Type":"ContainerStarted","Data":"d2deb0d160baeb47b927237374c8bdc0e14ea1e98dbc8f9fc56f5d1fb43f7dd9"}
Sep 16 16:41:28.956594 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:28.956185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lb6ps" event={"ID":"a0fcb09c-502d-4494-99fc-a128ca504f2e","Type":"ContainerStarted","Data":"edfdca08804c6cc477a3b4d2bd756065ab9705d83db9ff589a83eb739c517555"}
Sep 16 16:41:29.961171 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:29.961098    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lb6ps" event={"ID":"a0fcb09c-502d-4494-99fc-a128ca504f2e","Type":"ContainerStarted","Data":"15909fedaca5e5a517a171cfae6319fe68df0ce2eb228befa826e5f32edd44ec"}
Sep 16 16:41:31.226559 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:31.226509    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:41:31.969786 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:31.969753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lb6ps" event={"ID":"a0fcb09c-502d-4494-99fc-a128ca504f2e","Type":"ContainerStarted","Data":"106d062b328a80e8054fe9c0419b568fb6faa9ad84cd1ab6171186eb972bac23"}
Sep 16 16:41:31.988328 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:31.988285    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lb6ps" podStartSLOduration=33.498196277 podStartE2EDuration="35.988271364s" totalImagesPullingTime="2.490075087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 16:40:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:31.988066343 +0000 UTC m=+307.484939467" watchObservedRunningTime="2026-09-16 16:41:31.988271364 +0000 UTC m=+307.485144488"
Sep 16 16:41:32.891517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:32.891487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:32.971116 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:32.971089    2577 scope.go:117] "RemoveContainer" containerID="b749b8d59d1e12e339a36bd1c7926434bb191f805166575f56f2a836357b2f21"
Sep 16 16:41:32.971281 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:32.971246    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-696884fb9c-8trxf_openshift-console-operator(911d89a7-998b-43e4-a787-8c3f14775db3)\"" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podUID="911d89a7-998b-43e4-a787-8c3f14775db3"
Sep 16 16:41:32.991599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:32.991568    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:33.049266 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:33.049238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:34.164961 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:34.164916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:34.230819 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:34.230787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:34.973403 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:34.973374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:41:35.264912 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:35.264832    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:37.769621 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:37.769588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:37.960815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:37.960784    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:38.056694 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.056618    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hdwzx"]
Sep 16 16:41:38.058906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.058890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.066052 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.066030    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 16:41:38.066327 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.066311    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 16:41:38.066824 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.066808    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 16:41:38.066922 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.066878    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-284lx\""
Sep 16 16:41:38.067998 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.067983    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 16:41:38.101703 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.101678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:38.180542 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-textfile\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.180660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.180660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-metrics-client-ca\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.180660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-sys\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.180660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-root\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.180847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-tls\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.180847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df44j\" (UniqueName: \"kubernetes.io/projected/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-kube-api-access-df44j\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.180847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-accelerators-collector-config\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.180847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.180821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-wtmp\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.189666 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.189640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:38.281713 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-accelerators-collector-config\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.281853 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-wtmp\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.281893 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-textfile\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.281893 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.281959 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-metrics-client-ca\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.281959 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-sys\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.281959 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-wtmp\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.281959 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-root\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.282142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.281983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-tls\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.282142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.282050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-sys\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.282142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.282101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df44j\" (UniqueName: \"kubernetes.io/projected/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-kube-api-access-df44j\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.282142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.282053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-root\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.282334 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.282153    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-textfile\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.282334 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.282277    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-accelerators-collector-config\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.282499 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.282479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-metrics-client-ca\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.284350 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.284325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-tls\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.284487 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.284470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.291557 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.291522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df44j\" (UniqueName: \"kubernetes.io/projected/9d46271d-005e-4ebc-8ac4-6ac119cd0fb8-kube-api-access-df44j\") pod \"node-exporter-hdwzx\" (UID: \"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8\") " pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.367766 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.367707    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hdwzx"
Sep 16 16:41:38.377831 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:38.377804    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d46271d_005e_4ebc_8ac4_6ac119cd0fb8.slice/crio-7e28a79a8d3fcc979dd2d7ec4daaf4adf39a454e2526b3c174c9f36b2182b41a WatchSource:0}: Error finding container 7e28a79a8d3fcc979dd2d7ec4daaf4adf39a454e2526b3c174c9f36b2182b41a: Status 404 returned error can't find the container with id 7e28a79a8d3fcc979dd2d7ec4daaf4adf39a454e2526b3c174c9f36b2182b41a
Sep 16 16:41:38.827578 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.827545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:38.994737 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:38.994703    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hdwzx" event={"ID":"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8","Type":"ContainerStarted","Data":"7e28a79a8d3fcc979dd2d7ec4daaf4adf39a454e2526b3c174c9f36b2182b41a"}
Sep 16 16:41:39.093634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.093556    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:41:39.096300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.096278    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.098838 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.098789    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 16:41:39.098838 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.098802    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 16:41:39.099379 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.099172    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 16:41:39.099379 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.099223    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 16:41:39.099547 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.099440    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 16:41:39.099547 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.099477    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 16:41:39.099677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.099660    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 16:41:39.099760 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.099745    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 16:41:39.100125 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.099888    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 16:41:39.100322 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.100300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kz2wg\""
Sep 16 16:41:39.116226 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.116207    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:41:39.145899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.145873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:39.188129 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188261 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188261 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188261 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-web-config\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188649 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-config-volume\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188649 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188649 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-config-out\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188649 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.188649 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.188611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d58g9\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-kube-api-access-d58g9\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.289976 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.289940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.289983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-web-config\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-config-volume\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290157 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:39.290122    2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 16 16:41:39.290463 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:39.290195    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls podName:7931aa29-bffc-4464-8247-206ac98f6221 nodeName:}" failed. No retries permitted until 2026-09-16 16:41:39.790174504 +0000 UTC m=+315.287047620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7931aa29-bffc-4464-8247-206ac98f6221") : secret "alertmanager-main-tls" not found
Sep 16 16:41:39.290463 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290607 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290688 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-config-out\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290758 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290812 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290788    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d58g9\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-kube-api-access-d58g9\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290864 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290916 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.290968 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.290918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.291777 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.291750    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.293219 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.292867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.293992 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.293972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.294126 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.294051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.294682 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.294620    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.295028 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.294963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-web-config\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.295183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.295154    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.295977 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.295955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.296103 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.296073    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-config-volume\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.296373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.296353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-config-out\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.299627 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.299607    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d58g9\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-kube-api-access-d58g9\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.408321 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.408282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:39.510664 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.510639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:39.795113 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.795029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:39.797628 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:39.797604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:40.000786 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:40.000746    2577 generic.go:356] "Generic (PLEG): container finished" podID="9d46271d-005e-4ebc-8ac4-6ac119cd0fb8" containerID="b8f3da74262a41249700e90e52f1b7b71bacbd8394fb22aade424660f991dd3d" exitCode=0
Sep 16 16:41:40.001227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:40.000815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hdwzx" event={"ID":"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8","Type":"ContainerDied","Data":"b8f3da74262a41249700e90e52f1b7b71bacbd8394fb22aade424660f991dd3d"}
Sep 16 16:41:40.009269 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:40.009251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:41:40.294650 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:40.294543    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:41:40.299062 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:40.299001    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7931aa29_bffc_4464_8247_206ac98f6221.slice/crio-58ad535edc82cf8c6746d54f5cf2faa69966193d19169b94dd9e67880c266cef WatchSource:0}: Error finding container 58ad535edc82cf8c6746d54f5cf2faa69966193d19169b94dd9e67880c266cef: Status 404 returned error can't find the container with id 58ad535edc82cf8c6746d54f5cf2faa69966193d19169b94dd9e67880c266cef
Sep 16 16:41:41.006923 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:41.006892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerStarted","Data":"58ad535edc82cf8c6746d54f5cf2faa69966193d19169b94dd9e67880c266cef"}
Sep 16 16:41:41.007393 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:41.007185    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:41.009687 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:41.009662    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hdwzx" event={"ID":"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8","Type":"ContainerStarted","Data":"198b615bcc89782464a4802afc8d2c6415185f9b155cbac2afb0f989cf8e163c"}
Sep 16 16:41:41.009805 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:41.009708    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hdwzx" event={"ID":"9d46271d-005e-4ebc-8ac4-6ac119cd0fb8","Type":"ContainerStarted","Data":"d285fc8e488d4a2b6b6e8cba7e66b47397aebac55b09ecf060747f9fbe0e88ea"}
Sep 16 16:41:41.035977 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:41.035926    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hdwzx" podStartSLOduration=1.902999569 podStartE2EDuration="3.035909153s" totalImagesPullingTime="1.070103492s" totalInitContainerRuntime="62.806092ms" isStatefulPod=true podCreationTimestamp="2026-09-16 16:41:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:41.034042718 +0000 UTC m=+316.530915842" watchObservedRunningTime="2026-09-16 16:41:41.035909153 +0000 UTC m=+316.532782276"
Sep 16 16:41:41.312986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:41.312906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:41.384507 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:41.384462    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:42.014801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:42.014770    2577 generic.go:356] "Generic (PLEG): container finished" podID="7931aa29-bffc-4464-8247-206ac98f6221" containerID="b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d" exitCode=0
Sep 16 16:41:42.015152 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:42.014860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerDied","Data":"b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d"}
Sep 16 16:41:42.930180 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:42.930150    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67d8774cb9-gwd48"
Sep 16 16:41:43.158730 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.158685    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:43.250333 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.250254    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"]
Sep 16 16:41:43.252665 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.252629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:43.253102 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.253050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.255666 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.255647    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 16 16:41:43.255801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.255782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 16 16:41:43.255921 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.255874    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 16 16:41:43.256134 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.256109    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wgglk\""
Sep 16 16:41:43.256134 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.256122    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 16 16:41:43.256134 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.256133    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 16 16:41:43.267331 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.267084    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 16 16:41:43.273907 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.273881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"]
Sep 16 16:41:43.331438 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.331359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:43.426228 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.426196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-telemeter-client-tls\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.426416 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.426244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-secret-telemeter-client\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.426416 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.426315    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkbgg\" (UniqueName: \"kubernetes.io/projected/41c6d354-0a6d-47ab-b2b8-8447825922e1-kube-api-access-fkbgg\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.426416 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.426382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-federate-client-tls\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.426566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.426440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.426566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.426461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-metrics-client-ca\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.426646 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.426575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-serving-certs-ca-bundle\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.426646 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.426594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.527619 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.527515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-federate-client-tls\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.527619 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.527593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.527851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.527620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-metrics-client-ca\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.527851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.527686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-serving-certs-ca-bundle\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.527851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.527709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.527851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.527745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-telemeter-client-tls\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.527851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.527775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-secret-telemeter-client\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.527851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.527824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkbgg\" (UniqueName: \"kubernetes.io/projected/41c6d354-0a6d-47ab-b2b8-8447825922e1-kube-api-access-fkbgg\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.528762 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.528671    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-metrics-client-ca\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.528762 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.528676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-serving-certs-ca-bundle\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.528974 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.528936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41c6d354-0a6d-47ab-b2b8-8447825922e1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.530817 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.530790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-federate-client-tls\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.530920 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.530826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.530920 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.530900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-secret-telemeter-client\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.531392 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.531352    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/41c6d354-0a6d-47ab-b2b8-8447825922e1-telemeter-client-tls\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.536664 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.536641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkbgg\" (UniqueName: \"kubernetes.io/projected/41c6d354-0a6d-47ab-b2b8-8447825922e1-kube-api-access-fkbgg\") pod \"telemeter-client-55c797ccb7-8w7wg\" (UID: \"41c6d354-0a6d-47ab-b2b8-8447825922e1\") " pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.569384 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.569355    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"
Sep 16 16:41:43.894198 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:43.894175    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55c797ccb7-8w7wg"]
Sep 16 16:41:43.897654 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:43.897610    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c6d354_0a6d_47ab_b2b8_8447825922e1.slice/crio-a853a4d1fe7a9878179d38318caa108aa7bda169885deaf2d88cdf7e447fb416 WatchSource:0}: Error finding container a853a4d1fe7a9878179d38318caa108aa7bda169885deaf2d88cdf7e447fb416: Status 404 returned error can't find the container with id a853a4d1fe7a9878179d38318caa108aa7bda169885deaf2d88cdf7e447fb416
Sep 16 16:41:44.025996 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:44.025964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerStarted","Data":"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2"}
Sep 16 16:41:44.026180 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:44.026043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerStarted","Data":"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259"}
Sep 16 16:41:44.026180 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:44.026081    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerStarted","Data":"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3"}
Sep 16 16:41:44.026180 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:44.026119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerStarted","Data":"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a"}
Sep 16 16:41:44.026180 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:44.026154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerStarted","Data":"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50"}
Sep 16 16:41:44.027024 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:44.026998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg" event={"ID":"41c6d354-0a6d-47ab-b2b8-8447825922e1","Type":"ContainerStarted","Data":"a853a4d1fe7a9878179d38318caa108aa7bda169885deaf2d88cdf7e447fb416"}
Sep 16 16:41:44.259996 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:44.259957    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:45.033644 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:45.033604    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerStarted","Data":"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9"}
Sep 16 16:41:45.061771 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:45.061705    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.462413908 podStartE2EDuration="6.061681085s" totalImagesPullingTime="3.583337518s" totalInitContainerRuntime="15.929659ms" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:45.059880629 +0000 UTC m=+320.556753770" watchObservedRunningTime="2026-09-16 16:41:45.061681085 +0000 UTC m=+320.558554208"
Sep 16 16:41:45.852839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:45.852790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:45.932284 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:45.932246    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:46.100730 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.100692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:46.200848 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.200801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:46.239381 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.239355    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv" podUID="34007bd9-3504-45a4-9175-3582bffd83d6" containerName="registry" containerID="cri-o://8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610" gracePeriod=30
Sep 16 16:41:46.483083 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.483061    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:41:46.553135 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553107    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") pod \"34007bd9-3504-45a4-9175-3582bffd83d6\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") "
Sep 16 16:41:46.553305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553151    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-trusted-ca\") pod \"34007bd9-3504-45a4-9175-3582bffd83d6\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") "
Sep 16 16:41:46.553305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553198    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-installation-pull-secrets\") pod \"34007bd9-3504-45a4-9175-3582bffd83d6\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") "
Sep 16 16:41:46.553305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553231    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34007bd9-3504-45a4-9175-3582bffd83d6-ca-trust-extracted\") pod \"34007bd9-3504-45a4-9175-3582bffd83d6\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") "
Sep 16 16:41:46.553305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553256    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5tsx\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-kube-api-access-d5tsx\") pod \"34007bd9-3504-45a4-9175-3582bffd83d6\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") "
Sep 16 16:41:46.553510 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553409    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-bound-sa-token\") pod \"34007bd9-3504-45a4-9175-3582bffd83d6\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") "
Sep 16 16:41:46.553510 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553448    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-registry-certificates\") pod \"34007bd9-3504-45a4-9175-3582bffd83d6\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") "
Sep 16 16:41:46.553510 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553486    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-image-registry-private-configuration\") pod \"34007bd9-3504-45a4-9175-3582bffd83d6\" (UID: \"34007bd9-3504-45a4-9175-3582bffd83d6\") "
Sep 16 16:41:46.553699 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553581    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "34007bd9-3504-45a4-9175-3582bffd83d6" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:41:46.553842 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.553820    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-trusted-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:41:46.554109 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.554083    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "34007bd9-3504-45a4-9175-3582bffd83d6" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:41:46.555746 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.555716    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-kube-api-access-d5tsx" (OuterVolumeSpecName: "kube-api-access-d5tsx") pod "34007bd9-3504-45a4-9175-3582bffd83d6" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6"). InnerVolumeSpecName "kube-api-access-d5tsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:41:46.555833 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.555720    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "34007bd9-3504-45a4-9175-3582bffd83d6" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:41:46.555833 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.555790    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "34007bd9-3504-45a4-9175-3582bffd83d6" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:41:46.556083 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.556063    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "34007bd9-3504-45a4-9175-3582bffd83d6" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:41:46.556260 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.556243    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "34007bd9-3504-45a4-9175-3582bffd83d6" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:41:46.562042 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.562020    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34007bd9-3504-45a4-9175-3582bffd83d6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "34007bd9-3504-45a4-9175-3582bffd83d6" (UID: "34007bd9-3504-45a4-9175-3582bffd83d6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:41:46.654916 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.654887    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-registry-tls\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:41:46.654916 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.654912    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-installation-pull-secrets\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:41:46.654916 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.654922    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/34007bd9-3504-45a4-9175-3582bffd83d6-ca-trust-extracted\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:41:46.655097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.654930    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5tsx\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-kube-api-access-d5tsx\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:41:46.655097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.654939    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34007bd9-3504-45a4-9175-3582bffd83d6-bound-sa-token\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:41:46.655097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.654947    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/34007bd9-3504-45a4-9175-3582bffd83d6-registry-certificates\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:41:46.655097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:46.654955    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/34007bd9-3504-45a4-9175-3582bffd83d6-image-registry-private-configuration\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:41:47.043665 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.043630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg" event={"ID":"41c6d354-0a6d-47ab-b2b8-8447825922e1","Type":"ContainerStarted","Data":"bdc9ff7e89cabb8d3a682f1aeb516d9cc7b2ee7e7789b5d8d18c176c9a1720fc"}
Sep 16 16:41:47.044097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.043690    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg" event={"ID":"41c6d354-0a6d-47ab-b2b8-8447825922e1","Type":"ContainerStarted","Data":"443e60e37c56cf04fe051ec707a7cb3119c5f16e95081f3f897d1614802a5d7c"}
Sep 16 16:41:47.044097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.043716    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg" event={"ID":"41c6d354-0a6d-47ab-b2b8-8447825922e1","Type":"ContainerStarted","Data":"e7df65d3ef3ac9315c6b35f68d0ee4d596f834a5ad2b972b409162338a7f580d"}
Sep 16 16:41:47.045019 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.044995    2577 generic.go:356] "Generic (PLEG): container finished" podID="34007bd9-3504-45a4-9175-3582bffd83d6" containerID="8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610" exitCode=0
Sep 16 16:41:47.045122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.045034    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv"
Sep 16 16:41:47.045122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.045047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv" event={"ID":"34007bd9-3504-45a4-9175-3582bffd83d6","Type":"ContainerDied","Data":"8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610"}
Sep 16 16:41:47.045122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.045078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6788cc6fc8-6n9vv" event={"ID":"34007bd9-3504-45a4-9175-3582bffd83d6","Type":"ContainerDied","Data":"2daf848cbaf6510b0b0296c3d36dddcc96aff22167659dbf3c7577f111d6db0b"}
Sep 16 16:41:47.045122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.045100    2577 scope.go:117] "RemoveContainer" containerID="8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610"
Sep 16 16:41:47.053012 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.052995    2577 scope.go:117] "RemoveContainer" containerID="8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610"
Sep 16 16:41:47.053264 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:41:47.053242    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610\": container with ID starting with 8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610 not found: ID does not exist" containerID="8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610"
Sep 16 16:41:47.053324 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.053273    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610"} err="failed to get container status \"8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610\": rpc error: code = NotFound desc = could not find container \"8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610\": container with ID starting with 8a0b1d69f66d5d120dd86a14b5008e0f028a299aced21a48832c17a270203610 not found: ID does not exist"
Sep 16 16:41:47.067263 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.067228    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55c797ccb7-8w7wg" podStartSLOduration=1.790248768 podStartE2EDuration="4.067217157s" totalImagesPullingTime="2.276968389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:47.066125204 +0000 UTC m=+322.562998328" watchObservedRunningTime="2026-09-16 16:41:47.067217157 +0000 UTC m=+322.564090280"
Sep 16 16:41:47.083458 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.083435    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6788cc6fc8-6n9vv"]
Sep 16 16:41:47.091825 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.091806    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6788cc6fc8-6n9vv"]
Sep 16 16:41:47.971455 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:47.971428    2577 scope.go:117] "RemoveContainer" containerID="b749b8d59d1e12e339a36bd1c7926434bb191f805166575f56f2a836357b2f21"
Sep 16 16:41:48.051305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.051280    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:41:48.051752 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.051404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" event={"ID":"911d89a7-998b-43e4-a787-8c3f14775db3","Type":"ContainerStarted","Data":"26d986d50b5e0081a797af576f0208eb4a556411b92f131d879f565762c2fe5e"}
Sep 16 16:41:48.052089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.052067    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:41:48.053401 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.053373    2577 patch_prober.go:28] interesting pod/console-operator-696884fb9c-8trxf container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.10:8443/readyz\": dial tcp 10.133.0.10:8443: connect: connection refused" start-of-body=
Sep 16 16:41:48.053508 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.053426    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podUID="911d89a7-998b-43e4-a787-8c3f14775db3" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.10:8443/readyz\": dial tcp 10.133.0.10:8443: connect: connection refused"
Sep 16 16:41:48.068195 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.068137    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-8trxf" podStartSLOduration=52.850354067 podStartE2EDuration="56.068120194s" totalImagesPullingTime="3.217766127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:48.067670621 +0000 UTC m=+323.564543745" watchObservedRunningTime="2026-09-16 16:41:48.068120194 +0000 UTC m=+323.564993319"
Sep 16 16:41:48.660122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.660089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:48.748007 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.747970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:48.846781 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.846752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:48.976524 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:48.976453    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34007bd9-3504-45a4-9175-3582bffd83d6" path="/var/lib/kubelet/pods/34007bd9-3504-45a4-9175-3582bffd83d6/volumes"
Sep 16 16:41:49.062737 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:49.062705    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-8trxf"
Sep 16 16:41:49.081422 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:49.081394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:49.349639 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:49.349558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:49.550188 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:49.550153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:49.610778 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:49.610695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:49.712943 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:49.712911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:49.911046 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:49.911016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:49.968913 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:49.968880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:50.119034 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:50.119004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:50.320353 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:50.320277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:50.807015 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:50.806978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:51.331332 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:51.331297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:54.442388 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.442352    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-595cf6466c-rz8h9"]
Sep 16 16:41:54.442800 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.442694    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34007bd9-3504-45a4-9175-3582bffd83d6" containerName="registry"
Sep 16 16:41:54.442800 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.442709    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="34007bd9-3504-45a4-9175-3582bffd83d6" containerName="registry"
Sep 16 16:41:54.442800 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.442758    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="34007bd9-3504-45a4-9175-3582bffd83d6" containerName="registry"
Sep 16 16:41:54.446241 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.446223    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.449843 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.449819    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 16:41:54.449972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.449875    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4zz72\""
Sep 16 16:41:54.449972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.449876    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 16:41:54.449972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.449912    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 16:41:54.449972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.449875    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 16:41:54.449972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.449819    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 16:41:54.449972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.449882    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 16:41:54.450271 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.449958    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 16:41:54.457615 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.457597    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-595cf6466c-rz8h9"]
Sep 16 16:41:54.517659 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.517636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-serving-cert\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.517791 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.517671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-oauth-config\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.517791 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.517690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-console-config\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.517791 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.517709    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d476b\" (UniqueName: \"kubernetes.io/projected/6191398b-279b-4ff6-9618-13596328a35a-kube-api-access-d476b\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.517911 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.517788    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-oauth-serving-cert\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.517911 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.517821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-service-ca\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.619106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.619075    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-serving-cert\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.619276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.619116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-oauth-config\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.619276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.619147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-console-config\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.619276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.619167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d476b\" (UniqueName: \"kubernetes.io/projected/6191398b-279b-4ff6-9618-13596328a35a-kube-api-access-d476b\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.619276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.619195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-oauth-serving-cert\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.619276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.619214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-service-ca\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.620044 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.620019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-console-config\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.620169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.620019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-service-ca\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.620169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.620083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-oauth-serving-cert\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.621576 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.621548    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-oauth-config\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.621713 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.621697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-serving-cert\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.627018 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.626991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d476b\" (UniqueName: \"kubernetes.io/projected/6191398b-279b-4ff6-9618-13596328a35a-kube-api-access-d476b\") pod \"console-595cf6466c-rz8h9\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") " pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:54.755174 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:54.755107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:41:55.028110 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:55.028041    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-595cf6466c-rz8h9"]
Sep 16 16:41:55.030706 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:41:55.030676    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6191398b_279b_4ff6_9618_13596328a35a.slice/crio-b3ae3454eebaed3f44074ba57a5a356063bc791c7a5cdda111e651bb5f7b048d WatchSource:0}: Error finding container b3ae3454eebaed3f44074ba57a5a356063bc791c7a5cdda111e651bb5f7b048d: Status 404 returned error can't find the container with id b3ae3454eebaed3f44074ba57a5a356063bc791c7a5cdda111e651bb5f7b048d
Sep 16 16:41:55.080323 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:55.080294    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595cf6466c-rz8h9" event={"ID":"6191398b-279b-4ff6-9618-13596328a35a","Type":"ContainerStarted","Data":"b3ae3454eebaed3f44074ba57a5a356063bc791c7a5cdda111e651bb5f7b048d"}
Sep 16 16:41:56.239301 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:56.239258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:56.364121 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:56.364080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:41:59.097146 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:59.097110    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595cf6466c-rz8h9" event={"ID":"6191398b-279b-4ff6-9618-13596328a35a","Type":"ContainerStarted","Data":"425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24"}
Sep 16 16:41:59.118522 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:59.118476    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-595cf6466c-rz8h9" podStartSLOduration=1.767037263 podStartE2EDuration="5.118461689s" totalImagesPullingTime="3.351424426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:41:59.117100683 +0000 UTC m=+334.613973820" watchObservedRunningTime="2026-09-16 16:41:59.118461689 +0000 UTC m=+334.615334813"
Sep 16 16:41:59.935394 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:41:59.935349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:01.110893 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:01.110852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:01.172425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:01.172368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:02.109437 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:02.109402    2577 generic.go:356] "Generic (PLEG): container finished" podID="9fbdfae1-d3a8-47d0-971c-a3210c3db180" containerID="2b58431c9de0aacf541aacf51cdc633f8ca50187bd10d62c8e16d71d44cefd75" exitCode=0
Sep 16 16:42:02.109631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:02.109474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494" event={"ID":"9fbdfae1-d3a8-47d0-971c-a3210c3db180","Type":"ContainerDied","Data":"2b58431c9de0aacf541aacf51cdc633f8ca50187bd10d62c8e16d71d44cefd75"}
Sep 16 16:42:02.109795 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:02.109781    2577 scope.go:117] "RemoveContainer" containerID="2b58431c9de0aacf541aacf51cdc633f8ca50187bd10d62c8e16d71d44cefd75"
Sep 16 16:42:02.529239 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:02.529202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:02.583976 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:02.583948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:03.114607 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.114575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-m4494" event={"ID":"9fbdfae1-d3a8-47d0-971c-a3210c3db180","Type":"ContainerStarted","Data":"b32472be4f618300618d3b3be17a4194461ec85cbba6e7d70a189928b37eb4df"}
Sep 16 16:42:03.119414 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.119390    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-676d565c68-px9jn"]
Sep 16 16:42:03.122709 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.122693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.129796 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.129774    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 16:42:03.132756 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.132736    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-676d565c68-px9jn"]
Sep 16 16:42:03.194174 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.194144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-oauth-config\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.194358 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.194189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-serving-cert\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.194358 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.194212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-service-ca\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.194358 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.194278    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-oauth-serving-cert\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.194517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.194350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-console-config\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.194517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.194387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dps6b\" (UniqueName: \"kubernetes.io/projected/0e94b133-2b54-475c-8c88-b98a861f6527-kube-api-access-dps6b\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.194517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.194413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-trusted-ca-bundle\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.295135 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.295100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-serving-cert\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.295135 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.295134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-service-ca\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.295326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.295165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-oauth-serving-cert\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.295326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.295194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-console-config\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.295326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.295213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dps6b\" (UniqueName: \"kubernetes.io/projected/0e94b133-2b54-475c-8c88-b98a861f6527-kube-api-access-dps6b\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.295326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.295231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-trusted-ca-bundle\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.295326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.295266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-oauth-config\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.296036 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.296008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-console-config\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.296227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.296007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-service-ca\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.296227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.296036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-oauth-serving-cert\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.296412 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.296250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-trusted-ca-bundle\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.297675 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.297655    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-oauth-config\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.297942 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.297921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-serving-cert\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.302497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.302476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dps6b\" (UniqueName: \"kubernetes.io/projected/0e94b133-2b54-475c-8c88-b98a861f6527-kube-api-access-dps6b\") pod \"console-676d565c68-px9jn\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") " pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.431911 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.431882    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:03.705010 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:03.704908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-676d565c68-px9jn"]
Sep 16 16:42:03.707874 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:42:03.707832    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e94b133_2b54_475c_8c88_b98a861f6527.slice/crio-99b4e84f0c398ef360aa0bcdbf96be2ed8b7961f5d3ada10b596bb3f0d211582 WatchSource:0}: Error finding container 99b4e84f0c398ef360aa0bcdbf96be2ed8b7961f5d3ada10b596bb3f0d211582: Status 404 returned error can't find the container with id 99b4e84f0c398ef360aa0bcdbf96be2ed8b7961f5d3ada10b596bb3f0d211582
Sep 16 16:42:04.122169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:04.122048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676d565c68-px9jn" event={"ID":"0e94b133-2b54-475c-8c88-b98a861f6527","Type":"ContainerStarted","Data":"5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b"}
Sep 16 16:42:04.122169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:04.122124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676d565c68-px9jn" event={"ID":"0e94b133-2b54-475c-8c88-b98a861f6527","Type":"ContainerStarted","Data":"99b4e84f0c398ef360aa0bcdbf96be2ed8b7961f5d3ada10b596bb3f0d211582"}
Sep 16 16:42:04.139424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:04.139378    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-676d565c68-px9jn" podStartSLOduration=1.139364379 podStartE2EDuration="1.139364379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:42:04.138568228 +0000 UTC m=+339.635441353" watchObservedRunningTime="2026-09-16 16:42:04.139364379 +0000 UTC m=+339.636237503"
Sep 16 16:42:04.755595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:04.755551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:42:04.756131 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:04.755625    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:42:04.760293 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:04.760270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:42:05.129341 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:05.129264    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:42:05.728828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:05.728787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:05.793437 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:05.793395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:07.134979 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:07.134947    2577 generic.go:356] "Generic (PLEG): container finished" podID="5d100c30-471d-45a7-ae6e-4c4610cfb308" containerID="12757acbc8c9e9808da18e575694210c4043c70814005ca4bf2d4f05654d516d" exitCode=0
Sep 16 16:42:07.135354 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:07.135010    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7" event={"ID":"5d100c30-471d-45a7-ae6e-4c4610cfb308","Type":"ContainerDied","Data":"12757acbc8c9e9808da18e575694210c4043c70814005ca4bf2d4f05654d516d"}
Sep 16 16:42:07.135354 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:07.135290    2577 scope.go:117] "RemoveContainer" containerID="12757acbc8c9e9808da18e575694210c4043c70814005ca4bf2d4f05654d516d"
Sep 16 16:42:08.140450 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:08.140418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-jxzb7" event={"ID":"5d100c30-471d-45a7-ae6e-4c4610cfb308","Type":"ContainerStarted","Data":"aa911b28ecdd63b3d90e5b2cfbf9d1cf9d058d86f76dbc7fa80797353f013533"}
Sep 16 16:42:10.337337 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:10.337291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:10.396198 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:10.396173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:12.217055 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:12.217022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:12.656002 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:12.655971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:12.756799 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:12.756738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:13.432986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:13.432951    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:13.433414 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:13.433040    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:13.437785 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:13.437769    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:14.166294 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:14.166266    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:42:14.211807 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:14.211778    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-595cf6466c-rz8h9"]
Sep 16 16:42:14.542862 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:14.542777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:15.760855 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:15.760817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:15.817386 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:15.817335    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:16.098577 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:16.098431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:16.197664 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:16.197638    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:38.797875 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:38.797834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:42:38.800168 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:38.800143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07fb4aaa-bae1-4483-8001-e6bbd85f02e4-metrics-certs\") pod \"network-metrics-daemon-9wkss\" (UID: \"07fb4aaa-bae1-4483-8001-e6bbd85f02e4\") " pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:42:38.876702 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:38.876677    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v6lts\""
Sep 16 16:42:38.884828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:38.884808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9wkss"
Sep 16 16:42:39.157442 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.157414    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9wkss"]
Sep 16 16:42:39.160055 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:42:39.160027    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07fb4aaa_bae1_4483_8001_e6bbd85f02e4.slice/crio-b68a56f06ccc0fb8ebf7db9e9fcba25fb6fbf359aacc171d86ee0fc7b8b6c0d8 WatchSource:0}: Error finding container b68a56f06ccc0fb8ebf7db9e9fcba25fb6fbf359aacc171d86ee0fc7b8b6c0d8: Status 404 returned error can't find the container with id b68a56f06ccc0fb8ebf7db9e9fcba25fb6fbf359aacc171d86ee0fc7b8b6c0d8
Sep 16 16:42:39.231887 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.231834    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-595cf6466c-rz8h9" podUID="6191398b-279b-4ff6-9618-13596328a35a" containerName="console" containerID="cri-o://425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24" gracePeriod=15
Sep 16 16:42:39.260218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.260191    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9wkss" event={"ID":"07fb4aaa-bae1-4483-8001-e6bbd85f02e4","Type":"ContainerStarted","Data":"b68a56f06ccc0fb8ebf7db9e9fcba25fb6fbf359aacc171d86ee0fc7b8b6c0d8"}
Sep 16 16:42:39.450597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.450575    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:42:39.504012 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.503985    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-serving-cert\") pod \"6191398b-279b-4ff6-9618-13596328a35a\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") "
Sep 16 16:42:39.504155 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.504024    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-oauth-config\") pod \"6191398b-279b-4ff6-9618-13596328a35a\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") "
Sep 16 16:42:39.504155 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.504062    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d476b\" (UniqueName: \"kubernetes.io/projected/6191398b-279b-4ff6-9618-13596328a35a-kube-api-access-d476b\") pod \"6191398b-279b-4ff6-9618-13596328a35a\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") "
Sep 16 16:42:39.504269 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.504162    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-oauth-serving-cert\") pod \"6191398b-279b-4ff6-9618-13596328a35a\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") "
Sep 16 16:42:39.504269 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.504199    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-service-ca\") pod \"6191398b-279b-4ff6-9618-13596328a35a\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") "
Sep 16 16:42:39.504269 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.504229    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-console-config\") pod \"6191398b-279b-4ff6-9618-13596328a35a\" (UID: \"6191398b-279b-4ff6-9618-13596328a35a\") "
Sep 16 16:42:39.504618 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.504582    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-service-ca" (OuterVolumeSpecName: "service-ca") pod "6191398b-279b-4ff6-9618-13596328a35a" (UID: "6191398b-279b-4ff6-9618-13596328a35a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:42:39.504618 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.504576    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6191398b-279b-4ff6-9618-13596328a35a" (UID: "6191398b-279b-4ff6-9618-13596328a35a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:42:39.504782 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.504628    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-console-config" (OuterVolumeSpecName: "console-config") pod "6191398b-279b-4ff6-9618-13596328a35a" (UID: "6191398b-279b-4ff6-9618-13596328a35a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:42:39.506172 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.506152    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6191398b-279b-4ff6-9618-13596328a35a" (UID: "6191398b-279b-4ff6-9618-13596328a35a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:39.506218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.506175    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6191398b-279b-4ff6-9618-13596328a35a" (UID: "6191398b-279b-4ff6-9618-13596328a35a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:39.506218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.506185    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6191398b-279b-4ff6-9618-13596328a35a-kube-api-access-d476b" (OuterVolumeSpecName: "kube-api-access-d476b") pod "6191398b-279b-4ff6-9618-13596328a35a" (UID: "6191398b-279b-4ff6-9618-13596328a35a"). InnerVolumeSpecName "kube-api-access-d476b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:42:39.605280 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.605252    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-serving-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:39.605280 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.605276    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6191398b-279b-4ff6-9618-13596328a35a-console-oauth-config\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:39.605431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.605287    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d476b\" (UniqueName: \"kubernetes.io/projected/6191398b-279b-4ff6-9618-13596328a35a-kube-api-access-d476b\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:39.605431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.605296    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-oauth-serving-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:39.605431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.605304    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:39.605431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:39.605313    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6191398b-279b-4ff6-9618-13596328a35a-console-config\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:40.266495 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.266463    2577 generic.go:356] "Generic (PLEG): container finished" podID="6191398b-279b-4ff6-9618-13596328a35a" containerID="425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24" exitCode=0
Sep 16 16:42:40.266970 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.266516    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595cf6466c-rz8h9"
Sep 16 16:42:40.266970 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.266517    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595cf6466c-rz8h9" event={"ID":"6191398b-279b-4ff6-9618-13596328a35a","Type":"ContainerDied","Data":"425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24"}
Sep 16 16:42:40.266970 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.266651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595cf6466c-rz8h9" event={"ID":"6191398b-279b-4ff6-9618-13596328a35a","Type":"ContainerDied","Data":"b3ae3454eebaed3f44074ba57a5a356063bc791c7a5cdda111e651bb5f7b048d"}
Sep 16 16:42:40.266970 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.266675    2577 scope.go:117] "RemoveContainer" containerID="425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24"
Sep 16 16:42:40.279072 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.276745    2577 scope.go:117] "RemoveContainer" containerID="425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24"
Sep 16 16:42:40.279072 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:42:40.277057    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24\": container with ID starting with 425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24 not found: ID does not exist" containerID="425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24"
Sep 16 16:42:40.279072 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.277090    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24"} err="failed to get container status \"425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24\": rpc error: code = NotFound desc = could not find container \"425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24\": container with ID starting with 425e7af05058e61d61123dc98bb5a1fedcd96e1521966b99b7e09fa5cbdd9b24 not found: ID does not exist"
Sep 16 16:42:40.293297 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.293271    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-595cf6466c-rz8h9"]
Sep 16 16:42:40.296765 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.296744    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-595cf6466c-rz8h9"]
Sep 16 16:42:40.651411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.651377    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:40.752885 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.752854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:40.976189 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:40.976156    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6191398b-279b-4ff6-9618-13596328a35a" path="/var/lib/kubelet/pods/6191398b-279b-4ff6-9618-13596328a35a/volumes"
Sep 16 16:42:41.272995 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:41.272912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9wkss" event={"ID":"07fb4aaa-bae1-4483-8001-e6bbd85f02e4","Type":"ContainerStarted","Data":"af6c8d5d3ecbd7e7b4798e87a3700b2714c01425b64a8e9bc0ee4d31ebbf2f89"}
Sep 16 16:42:41.272995 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:41.272964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9wkss" event={"ID":"07fb4aaa-bae1-4483-8001-e6bbd85f02e4","Type":"ContainerStarted","Data":"1437367bb1b82fd62721e4087c7c091cf02275887b87303b64ffb9fbb3da97b9"}
Sep 16 16:42:41.290761 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:41.290713    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9wkss" podStartSLOduration=374.851581568 podStartE2EDuration="6m16.290698314s" totalImagesPullingTime="1.439116746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:42:41.289992662 +0000 UTC m=+376.786865786" watchObservedRunningTime="2026-09-16 16:42:41.290698314 +0000 UTC m=+376.787571419"
Sep 16 16:42:41.429641 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:41.429612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:41.529474 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:41.529395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:54.159838 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:54.159801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:54.219138 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:54.219099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:56.213366 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:56.213327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:56.277627 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:56.277592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:42:58.424273 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:58.424239    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:42:58.424655 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:58.424542    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="alertmanager" containerID="cri-o://4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50" gracePeriod=120
Sep 16 16:42:58.424655 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:58.424619    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy-web" containerID="cri-o://b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3" gracePeriod=120
Sep 16 16:42:58.424790 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:58.424664    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy" containerID="cri-o://70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259" gracePeriod=120
Sep 16 16:42:58.424790 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:58.424664    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="config-reloader" containerID="cri-o://ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a" gracePeriod=120
Sep 16 16:42:58.424790 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:58.424683    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="prom-label-proxy" containerID="cri-o://5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9" gracePeriod=120
Sep 16 16:42:58.424790 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:58.424611    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy-metric" containerID="cri-o://06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2" gracePeriod=120
Sep 16 16:42:59.344258 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.344222    2577 generic.go:356] "Generic (PLEG): container finished" podID="7931aa29-bffc-4464-8247-206ac98f6221" containerID="5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9" exitCode=0
Sep 16 16:42:59.344258 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.344243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerDied","Data":"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9"}
Sep 16 16:42:59.344472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.344310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerDied","Data":"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259"}
Sep 16 16:42:59.344472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.344252    2577 generic.go:356] "Generic (PLEG): container finished" podID="7931aa29-bffc-4464-8247-206ac98f6221" containerID="70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259" exitCode=0
Sep 16 16:42:59.344472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.344345    2577 generic.go:356] "Generic (PLEG): container finished" podID="7931aa29-bffc-4464-8247-206ac98f6221" containerID="ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a" exitCode=0
Sep 16 16:42:59.344472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.344354    2577 generic.go:356] "Generic (PLEG): container finished" podID="7931aa29-bffc-4464-8247-206ac98f6221" containerID="4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50" exitCode=0
Sep 16 16:42:59.344472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.344417    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerDied","Data":"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a"}
Sep 16 16:42:59.344472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.344454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerDied","Data":"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50"}
Sep 16 16:42:59.661329 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.661304    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:42:59.768474 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768436    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-trusted-ca-bundle\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.768678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768481    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-web-config\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.768678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768518    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-config-volume\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.768678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768564    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-main-db\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.768678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-cluster-tls-config\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.768678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768615    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.768678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768640    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-tls-assets\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.768678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768672    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.769023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768712    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.769023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768735    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-metrics-client-ca\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.769023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768769    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-web\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.769023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-config-out\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.769023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d58g9\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-kube-api-access-d58g9\") pod \"7931aa29-bffc-4464-8247-206ac98f6221\" (UID: \"7931aa29-bffc-4464-8247-206ac98f6221\") "
Sep 16 16:42:59.769023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768965    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:42:59.769312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.769179    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-main-db\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.769312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.768855    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:42:59.769312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.769249    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:42:59.771485 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.771454    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-config-volume" (OuterVolumeSpecName: "config-volume") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:59.771670 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.771622    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:42:59.771810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.771779    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:59.772063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.772037    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-config-out" (OuterVolumeSpecName: "config-out") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:42:59.772169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.772060    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:59.772451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.772426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-kube-api-access-d58g9" (OuterVolumeSpecName: "kube-api-access-d58g9") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "kube-api-access-d58g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:42:59.772806 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.772782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:59.773336 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.773315    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:59.776369 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.776331    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:59.782464 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.782443    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-web-config" (OuterVolumeSpecName: "web-config") pod "7931aa29-bffc-4464-8247-206ac98f6221" (UID: "7931aa29-bffc-4464-8247-206ac98f6221"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:42:59.869901 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869879    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.869901 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869901    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-web-config\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869911    2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869919    2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-cluster-tls-config\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869927    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869937    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-tls-assets\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869945    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869955    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-main-tls\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869963    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7931aa29-bffc-4464-8247-206ac98f6221-metrics-client-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869971    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7931aa29-bffc-4464-8247-206ac98f6221-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869979    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7931aa29-bffc-4464-8247-206ac98f6221-config-out\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:42:59.870004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:42:59.869987    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d58g9\" (UniqueName: \"kubernetes.io/projected/7931aa29-bffc-4464-8247-206ac98f6221-kube-api-access-d58g9\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:43:00.351588 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.351558    2577 generic.go:356] "Generic (PLEG): container finished" podID="7931aa29-bffc-4464-8247-206ac98f6221" containerID="06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2" exitCode=0
Sep 16 16:43:00.351588 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.351583    2577 generic.go:356] "Generic (PLEG): container finished" podID="7931aa29-bffc-4464-8247-206ac98f6221" containerID="b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3" exitCode=0
Sep 16 16:43:00.351802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.351622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerDied","Data":"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2"}
Sep 16 16:43:00.351802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.351635    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.351802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.351678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerDied","Data":"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3"}
Sep 16 16:43:00.351802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.351706    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7931aa29-bffc-4464-8247-206ac98f6221","Type":"ContainerDied","Data":"58ad535edc82cf8c6746d54f5cf2faa69966193d19169b94dd9e67880c266cef"}
Sep 16 16:43:00.351802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.351737    2577 scope.go:117] "RemoveContainer" containerID="5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9"
Sep 16 16:43:00.359586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.359561    2577 scope.go:117] "RemoveContainer" containerID="06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2"
Sep 16 16:43:00.367092 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.367072    2577 scope.go:117] "RemoveContainer" containerID="70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259"
Sep 16 16:43:00.373929 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.373914    2577 scope.go:117] "RemoveContainer" containerID="b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3"
Sep 16 16:43:00.380127 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.380110    2577 scope.go:117] "RemoveContainer" containerID="ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a"
Sep 16 16:43:00.386881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.386859    2577 scope.go:117] "RemoveContainer" containerID="4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50"
Sep 16 16:43:00.389455 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.389435    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:43:00.393900 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.393885    2577 scope.go:117] "RemoveContainer" containerID="b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d"
Sep 16 16:43:00.397136 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.397118    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:43:00.400884 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.400869    2577 scope.go:117] "RemoveContainer" containerID="5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9"
Sep 16 16:43:00.401157 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:43:00.401140    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9\": container with ID starting with 5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9 not found: ID does not exist" containerID="5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9"
Sep 16 16:43:00.401227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.401167    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9"} err="failed to get container status \"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9\": rpc error: code = NotFound desc = could not find container \"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9\": container with ID starting with 5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9 not found: ID does not exist"
Sep 16 16:43:00.401227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.401190    2577 scope.go:117] "RemoveContainer" containerID="06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2"
Sep 16 16:43:00.401463 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:43:00.401437    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2\": container with ID starting with 06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2 not found: ID does not exist" containerID="06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2"
Sep 16 16:43:00.401511 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.401460    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2"} err="failed to get container status \"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2\": rpc error: code = NotFound desc = could not find container \"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2\": container with ID starting with 06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2 not found: ID does not exist"
Sep 16 16:43:00.401511 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.401476    2577 scope.go:117] "RemoveContainer" containerID="70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259"
Sep 16 16:43:00.401756 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:43:00.401741    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259\": container with ID starting with 70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259 not found: ID does not exist" containerID="70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259"
Sep 16 16:43:00.401804 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.401760    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259"} err="failed to get container status \"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259\": rpc error: code = NotFound desc = could not find container \"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259\": container with ID starting with 70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259 not found: ID does not exist"
Sep 16 16:43:00.401804 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.401772    2577 scope.go:117] "RemoveContainer" containerID="b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3"
Sep 16 16:43:00.402031 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:43:00.402013    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3\": container with ID starting with b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3 not found: ID does not exist" containerID="b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3"
Sep 16 16:43:00.402092 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.402038    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3"} err="failed to get container status \"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3\": rpc error: code = NotFound desc = could not find container \"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3\": container with ID starting with b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3 not found: ID does not exist"
Sep 16 16:43:00.402092 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.402059    2577 scope.go:117] "RemoveContainer" containerID="ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a"
Sep 16 16:43:00.402308 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:43:00.402292    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a\": container with ID starting with ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a not found: ID does not exist" containerID="ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a"
Sep 16 16:43:00.402342 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.402313    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a"} err="failed to get container status \"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a\": rpc error: code = NotFound desc = could not find container \"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a\": container with ID starting with ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a not found: ID does not exist"
Sep 16 16:43:00.402342 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.402329    2577 scope.go:117] "RemoveContainer" containerID="4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50"
Sep 16 16:43:00.402548 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:43:00.402517    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50\": container with ID starting with 4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50 not found: ID does not exist" containerID="4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50"
Sep 16 16:43:00.402597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.402547    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50"} err="failed to get container status \"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50\": rpc error: code = NotFound desc = could not find container \"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50\": container with ID starting with 4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50 not found: ID does not exist"
Sep 16 16:43:00.402597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.402564    2577 scope.go:117] "RemoveContainer" containerID="b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d"
Sep 16 16:43:00.402787 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:43:00.402770    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d\": container with ID starting with b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d not found: ID does not exist" containerID="b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d"
Sep 16 16:43:00.402844 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.402794    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d"} err="failed to get container status \"b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d\": rpc error: code = NotFound desc = could not find container \"b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d\": container with ID starting with b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d not found: ID does not exist"
Sep 16 16:43:00.402844 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.402813    2577 scope.go:117] "RemoveContainer" containerID="5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9"
Sep 16 16:43:00.403027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403012    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9"} err="failed to get container status \"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9\": rpc error: code = NotFound desc = could not find container \"5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9\": container with ID starting with 5384a30faaec085eb1c80c6cf7ae8bc0d6e71416d949fffe2fa277b9bb10b6d9 not found: ID does not exist"
Sep 16 16:43:00.403071 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403027    2577 scope.go:117] "RemoveContainer" containerID="06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2"
Sep 16 16:43:00.403247 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403230    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2"} err="failed to get container status \"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2\": rpc error: code = NotFound desc = could not find container \"06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2\": container with ID starting with 06daaa30f8453fbe40ec926ad2d3abd4acb61890a040393ca8aba93cd9f3e3f2 not found: ID does not exist"
Sep 16 16:43:00.403310 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403248    2577 scope.go:117] "RemoveContainer" containerID="70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259"
Sep 16 16:43:00.403468 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403452    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259"} err="failed to get container status \"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259\": rpc error: code = NotFound desc = could not find container \"70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259\": container with ID starting with 70c803088ef886712e8f4ec086ba03ec1771e870bc48bed6e2a40fefb2080259 not found: ID does not exist"
Sep 16 16:43:00.403468 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403467    2577 scope.go:117] "RemoveContainer" containerID="b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3"
Sep 16 16:43:00.403694 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403676    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3"} err="failed to get container status \"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3\": rpc error: code = NotFound desc = could not find container \"b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3\": container with ID starting with b2e92298f3d8b6d4b8f06f86b8ad999157396e8ce823156924b586e9766902a3 not found: ID does not exist"
Sep 16 16:43:00.403756 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403695    2577 scope.go:117] "RemoveContainer" containerID="ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a"
Sep 16 16:43:00.403916 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403897    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a"} err="failed to get container status \"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a\": rpc error: code = NotFound desc = could not find container \"ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a\": container with ID starting with ac1e27bed122dd59e40e164698ee755c31e5f671c94237ee48dcf10d326ee98a not found: ID does not exist"
Sep 16 16:43:00.403963 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.403916    2577 scope.go:117] "RemoveContainer" containerID="4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50"
Sep 16 16:43:00.404139 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.404122    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50"} err="failed to get container status \"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50\": rpc error: code = NotFound desc = could not find container \"4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50\": container with ID starting with 4ed0b6cc91c676f7aabf4f4fb58747417b0b02b565fcbcaa803ad2013b995e50 not found: ID does not exist"
Sep 16 16:43:00.404184 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.404141    2577 scope.go:117] "RemoveContainer" containerID="b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d"
Sep 16 16:43:00.404388 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.404365    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d"} err="failed to get container status \"b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d\": rpc error: code = NotFound desc = could not find container \"b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d\": container with ID starting with b478c957d6e7214e6e6e22b1082e867aea1e65e36deacb36191ce8c4f77c990d not found: ID does not exist"
Sep 16 16:43:00.461805 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.461780    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:43:00.462041 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462030    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="config-reloader"
Sep 16 16:43:00.462079 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462042    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="config-reloader"
Sep 16 16:43:00.462079 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462074    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6191398b-279b-4ff6-9618-13596328a35a" containerName="console"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462080    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6191398b-279b-4ff6-9618-13596328a35a" containerName="console"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462089    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="init-config-reloader"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462095    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="init-config-reloader"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462102    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462107    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462115    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="alertmanager"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462120    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="alertmanager"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462128    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy-metric"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462134    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy-metric"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462142    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy-web"
Sep 16 16:43:00.462144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462148    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy-web"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462157    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="prom-label-proxy"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462162    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="prom-label-proxy"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462219    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy-web"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462227    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462234    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="config-reloader"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462241    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="prom-label-proxy"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462248    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6191398b-279b-4ff6-9618-13596328a35a" containerName="console"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462254    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="alertmanager"
Sep 16 16:43:00.462443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.462261    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7931aa29-bffc-4464-8247-206ac98f6221" containerName="kube-rbac-proxy-metric"
Sep 16 16:43:00.465953 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.465939    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.475448 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.475422    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 16:43:00.475578 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.475430    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 16:43:00.475578 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.475486    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kz2wg\""
Sep 16 16:43:00.475694 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.475605    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 16:43:00.475902 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.475885    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 16:43:00.476047 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.476034    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 16:43:00.476206 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.476190    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 16:43:00.476276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.476229    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 16:43:00.476276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.476235    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 16:43:00.487278 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.487261    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 16:43:00.509454 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.509434    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:43:00.574268 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574370 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574370 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574296    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574370 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574315    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdbl9\" (UniqueName: \"kubernetes.io/projected/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-kube-api-access-fdbl9\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574370 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574356    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-config-out\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-config-volume\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-web-config\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.574683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.574650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.675671 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675685    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdbl9\" (UniqueName: \"kubernetes.io/projected/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-kube-api-access-fdbl9\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-config-out\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-config-volume\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-web-config\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675937    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.675970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.676003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.676044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.676063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.676066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.677014 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.676991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.677165 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.677046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.677645 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.677618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.678776 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.678752    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.678870 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.678818    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.678955 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.678937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.679304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.679274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-config-out\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.679638 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.679613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.679733 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.679664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.679862 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.679835    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-web-config\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.679926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.679896    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.680473 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.680457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-config-volume\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.684327 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.684311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdbl9\" (UniqueName: \"kubernetes.io/projected/dfca1cf2-c60c-495a-9661-acc2c9a9f8de-kube-api-access-fdbl9\") pod \"alertmanager-main-0\" (UID: \"dfca1cf2-c60c-495a-9661-acc2c9a9f8de\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.773818 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.773769    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 16:43:00.979433 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:00.979347    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7931aa29-bffc-4464-8247-206ac98f6221" path="/var/lib/kubelet/pods/7931aa29-bffc-4464-8247-206ac98f6221/volumes"
Sep 16 16:43:01.060423 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.060364    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 16:43:01.063230 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:43:01.063206    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfca1cf2_c60c_495a_9661_acc2c9a9f8de.slice/crio-a0e3a7844ea5efc9f13fea464b60164885c56747f6bbe2b7b0fae8b85a1a8274 WatchSource:0}: Error finding container a0e3a7844ea5efc9f13fea464b60164885c56747f6bbe2b7b0fae8b85a1a8274: Status 404 returned error can't find the container with id a0e3a7844ea5efc9f13fea464b60164885c56747f6bbe2b7b0fae8b85a1a8274
Sep 16 16:43:01.357337 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.357265    2577 generic.go:356] "Generic (PLEG): container finished" podID="dfca1cf2-c60c-495a-9661-acc2c9a9f8de" containerID="10e6b6f1077654088d691f2f4d9fcb79f17483885b4f0b0ea834ec37db50b27d" exitCode=0
Sep 16 16:43:01.357466 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.357353    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dfca1cf2-c60c-495a-9661-acc2c9a9f8de","Type":"ContainerDied","Data":"10e6b6f1077654088d691f2f4d9fcb79f17483885b4f0b0ea834ec37db50b27d"}
Sep 16 16:43:01.357466 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.357431    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dfca1cf2-c60c-495a-9661-acc2c9a9f8de","Type":"ContainerStarted","Data":"a0e3a7844ea5efc9f13fea464b60164885c56747f6bbe2b7b0fae8b85a1a8274"}
Sep 16 16:43:01.774685 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.774656    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-576c4478fc-vqs76"]
Sep 16 16:43:01.776844 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.776819    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.791802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.791782    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-576c4478fc-vqs76"]
Sep 16 16:43:01.833477 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.833454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:01.887442 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.887421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtm56\" (UniqueName: \"kubernetes.io/projected/d8efa0b5-6558-4814-8522-ccf0adc50529-kube-api-access-vtm56\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.887573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.887448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-oauth-serving-cert\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.887573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.887470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-serving-cert\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.887573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.887548    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-console-config\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.887683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.887580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-trusted-ca-bundle\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.887683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.887603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-service-ca\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.887683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.887621    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-oauth-config\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.914986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.914943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:01.988477 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.988452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-serving-cert\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.988605 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.988506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-console-config\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.988605 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.988560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-trusted-ca-bundle\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.988605 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.988595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-service-ca\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.988717 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.988624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-oauth-config\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.988717 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.988671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtm56\" (UniqueName: \"kubernetes.io/projected/d8efa0b5-6558-4814-8522-ccf0adc50529-kube-api-access-vtm56\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.988717 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.988698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-oauth-serving-cert\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.989445 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.989420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-service-ca\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.989580 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.989457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-console-config\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.989580 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.989497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-trusted-ca-bundle\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.989580 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.989552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-oauth-serving-cert\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.991061 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.991029    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-serving-cert\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.991061 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.991050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-oauth-config\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:01.997787 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:01.997767    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtm56\" (UniqueName: \"kubernetes.io/projected/d8efa0b5-6558-4814-8522-ccf0adc50529-kube-api-access-vtm56\") pod \"console-576c4478fc-vqs76\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") " pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:02.086696 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.086675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:02.356601 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.356428    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-576c4478fc-vqs76"]
Sep 16 16:43:02.360819 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:43:02.360786    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8efa0b5_6558_4814_8522_ccf0adc50529.slice/crio-c55528ca7795068363ee4897a3c43b5c6a24e88f412015ac02e3576d871a25cf WatchSource:0}: Error finding container c55528ca7795068363ee4897a3c43b5c6a24e88f412015ac02e3576d871a25cf: Status 404 returned error can't find the container with id c55528ca7795068363ee4897a3c43b5c6a24e88f412015ac02e3576d871a25cf
Sep 16 16:43:02.367883 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.367859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dfca1cf2-c60c-495a-9661-acc2c9a9f8de","Type":"ContainerStarted","Data":"fa5113c071a7d516c8dd40cdcd2a2cd4bd80ed41d127e8ec3a60d916ef8215cc"}
Sep 16 16:43:02.367954 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.367918    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dfca1cf2-c60c-495a-9661-acc2c9a9f8de","Type":"ContainerStarted","Data":"ad1dbafe9437a7d6ce73f8aecaa9a7d5493f22f876b03d0543e87a19b09f2916"}
Sep 16 16:43:02.367954 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.367946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dfca1cf2-c60c-495a-9661-acc2c9a9f8de","Type":"ContainerStarted","Data":"69f0df8ddf71ffd10490946ba5958136fa7885dc9cfcb69bcc00f3d6669c47b1"}
Sep 16 16:43:02.368043 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.367970    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dfca1cf2-c60c-495a-9661-acc2c9a9f8de","Type":"ContainerStarted","Data":"36af223e3bb5a1f350c812c5a5e85e7d8f054bed8f572697073e93dece2ce958"}
Sep 16 16:43:02.368043 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.367992    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dfca1cf2-c60c-495a-9661-acc2c9a9f8de","Type":"ContainerStarted","Data":"9bd325fe17558a2037138829e704ba674bb13611db9734696ff289dc0a87745f"}
Sep 16 16:43:02.368043 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.368025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dfca1cf2-c60c-495a-9661-acc2c9a9f8de","Type":"ContainerStarted","Data":"224f9693046a13352abfb34fbbe039d84c5bcb280080a6038f7de394671841fe"}
Sep 16 16:43:02.392975 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:02.392936    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.3741653019999998 podStartE2EDuration="2.39292108s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.755778ms" isStatefulPod=false podCreationTimestamp="2026-09-16 16:43:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:43:02.391043486 +0000 UTC m=+397.887916607" watchObservedRunningTime="2026-09-16 16:43:02.39292108 +0000 UTC m=+397.889794205"
Sep 16 16:43:03.373201 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:03.373159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c4478fc-vqs76" event={"ID":"d8efa0b5-6558-4814-8522-ccf0adc50529","Type":"ContainerStarted","Data":"c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05"}
Sep 16 16:43:03.373675 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:03.373221    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c4478fc-vqs76" event={"ID":"d8efa0b5-6558-4814-8522-ccf0adc50529","Type":"ContainerStarted","Data":"c55528ca7795068363ee4897a3c43b5c6a24e88f412015ac02e3576d871a25cf"}
Sep 16 16:43:03.390539 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:03.390489    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-576c4478fc-vqs76" podStartSLOduration=2.390473476 podStartE2EDuration="2.390473476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:43:03.389477164 +0000 UTC m=+398.886350303" watchObservedRunningTime="2026-09-16 16:43:03.390473476 +0000 UTC m=+398.887346600"
Sep 16 16:43:05.991965 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:05.991925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:06.050509 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:06.050472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:11.820428 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:11.820383    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:11.898425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:11.898371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:12.087427 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:12.087346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:12.087427 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:12.087409    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:12.092062 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:12.092040    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:12.413250 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:12.413223    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:43:12.478091 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:12.478050    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-676d565c68-px9jn"]
Sep 16 16:43:12.614067 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:12.614032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:12.713558 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:12.713449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:12.813574 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:12.813514    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:16.112439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:16.112405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:16.209956 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:16.209921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:30.817199 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:30.817162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:31.030097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:31.030060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:37.498424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.498383    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-676d565c68-px9jn" podUID="0e94b133-2b54-475c-8c88-b98a861f6527" containerName="console" containerID="cri-o://5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b" gracePeriod=15
Sep 16 16:43:37.731774 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.731748    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:43:37.869778 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.869706    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-serving-cert\") pod \"0e94b133-2b54-475c-8c88-b98a861f6527\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") "
Sep 16 16:43:37.869778 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.869739    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dps6b\" (UniqueName: \"kubernetes.io/projected/0e94b133-2b54-475c-8c88-b98a861f6527-kube-api-access-dps6b\") pod \"0e94b133-2b54-475c-8c88-b98a861f6527\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") "
Sep 16 16:43:37.869778 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.869774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-service-ca\") pod \"0e94b133-2b54-475c-8c88-b98a861f6527\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") "
Sep 16 16:43:37.870042 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.869804    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-console-config\") pod \"0e94b133-2b54-475c-8c88-b98a861f6527\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") "
Sep 16 16:43:37.870042 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.869834    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-trusted-ca-bundle\") pod \"0e94b133-2b54-475c-8c88-b98a861f6527\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") "
Sep 16 16:43:37.870042 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.869894    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-oauth-config\") pod \"0e94b133-2b54-475c-8c88-b98a861f6527\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") "
Sep 16 16:43:37.870042 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.869916    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-oauth-serving-cert\") pod \"0e94b133-2b54-475c-8c88-b98a861f6527\" (UID: \"0e94b133-2b54-475c-8c88-b98a861f6527\") "
Sep 16 16:43:37.870237 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.870170    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e94b133-2b54-475c-8c88-b98a861f6527" (UID: "0e94b133-2b54-475c-8c88-b98a861f6527"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:37.870237 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.870223    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-console-config" (OuterVolumeSpecName: "console-config") pod "0e94b133-2b54-475c-8c88-b98a861f6527" (UID: "0e94b133-2b54-475c-8c88-b98a861f6527"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:37.870354 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.870268    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e94b133-2b54-475c-8c88-b98a861f6527" (UID: "0e94b133-2b54-475c-8c88-b98a861f6527"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:37.870457 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.870433    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e94b133-2b54-475c-8c88-b98a861f6527" (UID: "0e94b133-2b54-475c-8c88-b98a861f6527"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:43:37.871940 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.871911    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e94b133-2b54-475c-8c88-b98a861f6527-kube-api-access-dps6b" (OuterVolumeSpecName: "kube-api-access-dps6b") pod "0e94b133-2b54-475c-8c88-b98a861f6527" (UID: "0e94b133-2b54-475c-8c88-b98a861f6527"). InnerVolumeSpecName "kube-api-access-dps6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:43:37.872031 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.871948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e94b133-2b54-475c-8c88-b98a861f6527" (UID: "0e94b133-2b54-475c-8c88-b98a861f6527"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:37.872106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.871961    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e94b133-2b54-475c-8c88-b98a861f6527" (UID: "0e94b133-2b54-475c-8c88-b98a861f6527"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:43:37.970700 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.970660    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-oauth-config\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:43:37.970810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.970736    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-oauth-serving-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:43:37.970810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.970756    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e94b133-2b54-475c-8c88-b98a861f6527-console-serving-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:43:37.970810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.970770    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dps6b\" (UniqueName: \"kubernetes.io/projected/0e94b133-2b54-475c-8c88-b98a861f6527-kube-api-access-dps6b\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:43:37.970810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.970786    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:43:37.970810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.970805    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-console-config\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:43:37.970965 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:37.970820    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e94b133-2b54-475c-8c88-b98a861f6527-trusted-ca-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:43:38.510399 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.510363    2577 generic.go:356] "Generic (PLEG): container finished" podID="0e94b133-2b54-475c-8c88-b98a861f6527" containerID="5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b" exitCode=0
Sep 16 16:43:38.510807 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.510418    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-676d565c68-px9jn"
Sep 16 16:43:38.510807 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.510451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676d565c68-px9jn" event={"ID":"0e94b133-2b54-475c-8c88-b98a861f6527","Type":"ContainerDied","Data":"5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b"}
Sep 16 16:43:38.510807 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.510499    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-676d565c68-px9jn" event={"ID":"0e94b133-2b54-475c-8c88-b98a861f6527","Type":"ContainerDied","Data":"99b4e84f0c398ef360aa0bcdbf96be2ed8b7961f5d3ada10b596bb3f0d211582"}
Sep 16 16:43:38.510807 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.510521    2577 scope.go:117] "RemoveContainer" containerID="5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b"
Sep 16 16:43:38.519175 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.519152    2577 scope.go:117] "RemoveContainer" containerID="5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b"
Sep 16 16:43:38.519446 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:43:38.519427    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b\": container with ID starting with 5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b not found: ID does not exist" containerID="5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b"
Sep 16 16:43:38.519488 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.519455    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b"} err="failed to get container status \"5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b\": rpc error: code = NotFound desc = could not find container \"5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b\": container with ID starting with 5501d0e19c97de53758db1b14c6524ea3cf9b0597e2f48655eabbd6dfd855e1b not found: ID does not exist"
Sep 16 16:43:38.534080 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.534057    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-676d565c68-px9jn"]
Sep 16 16:43:38.545728 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.545707    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-676d565c68-px9jn"]
Sep 16 16:43:38.975621 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:38.975591    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e94b133-2b54-475c-8c88-b98a861f6527" path="/var/lib/kubelet/pods/0e94b133-2b54-475c-8c88-b98a861f6527/volumes"
Sep 16 16:43:47.400265 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:47.400145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:43:47.537716 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:43:47.537674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:02.853915 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:02.853815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:02.955313 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:02.955278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:03.889433 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.889404    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"]
Sep 16 16:44:03.889818 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.889724    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e94b133-2b54-475c-8c88-b98a861f6527" containerName="console"
Sep 16 16:44:03.889818 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.889737    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e94b133-2b54-475c-8c88-b98a861f6527" containerName="console"
Sep 16 16:44:03.889818 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.889793    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e94b133-2b54-475c-8c88-b98a861f6527" containerName="console"
Sep 16 16:44:03.893052 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.893036    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:03.897741 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.897719    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 16:44:03.897845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.897802    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zbjqr\""
Sep 16 16:44:03.897845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.897722    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 16:44:03.909839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.909817    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"]
Sep 16 16:44:03.973811 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.973786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:03.973907 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.973819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zrmw\" (UniqueName: \"kubernetes.io/projected/ee00d82d-ad69-4d84-9532-11570cd3498b-kube-api-access-6zrmw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:03.973907 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:03.973844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:04.075132 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.075102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:04.075276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.075169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:04.075276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.075197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zrmw\" (UniqueName: \"kubernetes.io/projected/ee00d82d-ad69-4d84-9532-11570cd3498b-kube-api-access-6zrmw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:04.075482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.075460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:04.075574 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.075550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:04.083513 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.083492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zrmw\" (UniqueName: \"kubernetes.io/projected/ee00d82d-ad69-4d84-9532-11570cd3498b-kube-api-access-6zrmw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:04.202008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.201908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:04.477091 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.474630    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"]
Sep 16 16:44:04.480675 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:44:04.480648    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee00d82d_ad69_4d84_9532_11570cd3498b.slice/crio-f0fe58c987e5e636d9c3420ba9135b5da7bf86eec463d16334291dd4ca2bfb6a WatchSource:0}: Error finding container f0fe58c987e5e636d9c3420ba9135b5da7bf86eec463d16334291dd4ca2bfb6a: Status 404 returned error can't find the container with id f0fe58c987e5e636d9c3420ba9135b5da7bf86eec463d16334291dd4ca2bfb6a
Sep 16 16:44:04.613390 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:04.613354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7" event={"ID":"ee00d82d-ad69-4d84-9532-11570cd3498b","Type":"ContainerStarted","Data":"f0fe58c987e5e636d9c3420ba9135b5da7bf86eec463d16334291dd4ca2bfb6a"}
Sep 16 16:44:11.645444 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:11.645405    2577 generic.go:356] "Generic (PLEG): container finished" podID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerID="598728149eb8c113d176b5555ee7e0dd6e92f9f9a76358ac6778643ebf811544" exitCode=0
Sep 16 16:44:11.645881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:11.645479    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7" event={"ID":"ee00d82d-ad69-4d84-9532-11570cd3498b","Type":"ContainerDied","Data":"598728149eb8c113d176b5555ee7e0dd6e92f9f9a76358ac6778643ebf811544"}
Sep 16 16:44:15.664606 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:15.664577    2577 generic.go:356] "Generic (PLEG): container finished" podID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerID="a6e4c8f8d02e156adf9c3bb66fccb7a2f6549d5c74895fd429af0743e5970741" exitCode=0
Sep 16 16:44:15.664957 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:15.664661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7" event={"ID":"ee00d82d-ad69-4d84-9532-11570cd3498b","Type":"ContainerDied","Data":"a6e4c8f8d02e156adf9c3bb66fccb7a2f6549d5c74895fd429af0743e5970741"}
Sep 16 16:44:22.696348 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:22.696312    2577 generic.go:356] "Generic (PLEG): container finished" podID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerID="b0664b6406fcefa12b2992b8f3813b2a85d1ece533d78e7de2859013f1b9eb9d" exitCode=0
Sep 16 16:44:22.696802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:22.696396    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7" event={"ID":"ee00d82d-ad69-4d84-9532-11570cd3498b","Type":"ContainerDied","Data":"b0664b6406fcefa12b2992b8f3813b2a85d1ece533d78e7de2859013f1b9eb9d"}
Sep 16 16:44:23.826688 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:23.826664    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:23.949199 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:23.949168    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-util\") pod \"ee00d82d-ad69-4d84-9532-11570cd3498b\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") "
Sep 16 16:44:23.949346 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:23.949220    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-bundle\") pod \"ee00d82d-ad69-4d84-9532-11570cd3498b\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") "
Sep 16 16:44:23.949346 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:23.949257    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zrmw\" (UniqueName: \"kubernetes.io/projected/ee00d82d-ad69-4d84-9532-11570cd3498b-kube-api-access-6zrmw\") pod \"ee00d82d-ad69-4d84-9532-11570cd3498b\" (UID: \"ee00d82d-ad69-4d84-9532-11570cd3498b\") "
Sep 16 16:44:23.949954 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:23.949926    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-bundle" (OuterVolumeSpecName: "bundle") pod "ee00d82d-ad69-4d84-9532-11570cd3498b" (UID: "ee00d82d-ad69-4d84-9532-11570cd3498b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:23.951470 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:23.951416    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee00d82d-ad69-4d84-9532-11570cd3498b-kube-api-access-6zrmw" (OuterVolumeSpecName: "kube-api-access-6zrmw") pod "ee00d82d-ad69-4d84-9532-11570cd3498b" (UID: "ee00d82d-ad69-4d84-9532-11570cd3498b"). InnerVolumeSpecName "kube-api-access-6zrmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:44:23.953623 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:23.953603    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-util" (OuterVolumeSpecName: "util") pod "ee00d82d-ad69-4d84-9532-11570cd3498b" (UID: "ee00d82d-ad69-4d84-9532-11570cd3498b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:24.050701 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:24.050669    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-util\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:44:24.050701 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:24.050695    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee00d82d-ad69-4d84-9532-11570cd3498b-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:44:24.050701 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:24.050706    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zrmw\" (UniqueName: \"kubernetes.io/projected/ee00d82d-ad69-4d84-9532-11570cd3498b-kube-api-access-6zrmw\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:44:24.706706 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:24.706671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7" event={"ID":"ee00d82d-ad69-4d84-9532-11570cd3498b","Type":"ContainerDied","Data":"f0fe58c987e5e636d9c3420ba9135b5da7bf86eec463d16334291dd4ca2bfb6a"}
Sep 16 16:44:24.706867 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:24.706718    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540d2vf7"
Sep 16 16:44:24.706867 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:24.706721    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0fe58c987e5e636d9c3420ba9135b5da7bf86eec463d16334291dd4ca2bfb6a"
Sep 16 16:44:30.223815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:30.223780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:30.545194 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:30.545101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:32.262987 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.262952    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"]
Sep 16 16:44:32.263429 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.263406    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerName="pull"
Sep 16 16:44:32.263429 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.263428    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerName="pull"
Sep 16 16:44:32.263629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.263451    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerName="util"
Sep 16 16:44:32.263629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.263458    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerName="util"
Sep 16 16:44:32.263629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.263476    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerName="extract"
Sep 16 16:44:32.263629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.263484    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerName="extract"
Sep 16 16:44:32.263629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.263573    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee00d82d-ad69-4d84-9532-11570cd3498b" containerName="extract"
Sep 16 16:44:32.267483 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.267465    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"
Sep 16 16:44:32.270962 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.270942    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7mlxn\""
Sep 16 16:44:32.272835 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.272819    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:44:32.273208 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.273192    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 16 16:44:32.312550 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.312515    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"]
Sep 16 16:44:32.416348 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.416316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/494b933d-9df6-49ec-9702-75f6555818bf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ht8q\" (UID: \"494b933d-9df6-49ec-9702-75f6555818bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"
Sep 16 16:44:32.416348 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.416352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-685mx\" (UniqueName: \"kubernetes.io/projected/494b933d-9df6-49ec-9702-75f6555818bf-kube-api-access-685mx\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ht8q\" (UID: \"494b933d-9df6-49ec-9702-75f6555818bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"
Sep 16 16:44:32.517411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.517333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-685mx\" (UniqueName: \"kubernetes.io/projected/494b933d-9df6-49ec-9702-75f6555818bf-kube-api-access-685mx\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ht8q\" (UID: \"494b933d-9df6-49ec-9702-75f6555818bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"
Sep 16 16:44:32.517580 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.517431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/494b933d-9df6-49ec-9702-75f6555818bf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ht8q\" (UID: \"494b933d-9df6-49ec-9702-75f6555818bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"
Sep 16 16:44:32.517842 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.517823    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/494b933d-9df6-49ec-9702-75f6555818bf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ht8q\" (UID: \"494b933d-9df6-49ec-9702-75f6555818bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"
Sep 16 16:44:32.530936 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.530904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-685mx\" (UniqueName: \"kubernetes.io/projected/494b933d-9df6-49ec-9702-75f6555818bf-kube-api-access-685mx\") pod \"cert-manager-operator-controller-manager-55c444bf86-5ht8q\" (UID: \"494b933d-9df6-49ec-9702-75f6555818bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"
Sep 16 16:44:32.575741 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.575706    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"
Sep 16 16:44:32.853192 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:32.853169    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q"]
Sep 16 16:44:32.855136 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:44:32.855111    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494b933d_9df6_49ec_9702_75f6555818bf.slice/crio-c8b2e743365251fa2726320168efe775601aadf48e3f45a816c0ecf351e60fc2 WatchSource:0}: Error finding container c8b2e743365251fa2726320168efe775601aadf48e3f45a816c0ecf351e60fc2: Status 404 returned error can't find the container with id c8b2e743365251fa2726320168efe775601aadf48e3f45a816c0ecf351e60fc2
Sep 16 16:44:33.742520 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:33.742481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q" event={"ID":"494b933d-9df6-49ec-9702-75f6555818bf","Type":"ContainerStarted","Data":"c8b2e743365251fa2726320168efe775601aadf48e3f45a816c0ecf351e60fc2"}
Sep 16 16:44:34.061264 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:34.061173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:34.165156 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:34.165121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:35.734431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:35.734391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:35.753629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:35.753594    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q" event={"ID":"494b933d-9df6-49ec-9702-75f6555818bf","Type":"ContainerStarted","Data":"6fa695901db5ba58c25208f5668495bf0844d3b3d63fdd622d1821a472ac4fd3"}
Sep 16 16:44:35.814673 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:35.814429    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5ht8q" podStartSLOduration=1.619108156 podStartE2EDuration="3.814409751s" totalImagesPullingTime="2.195301595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:44:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:44:35.814095206 +0000 UTC m=+491.310968331" watchObservedRunningTime="2026-09-16 16:44:35.814409751 +0000 UTC m=+491.311282876"
Sep 16 16:44:35.859123 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:35.859089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:36.180228 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.180169    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:36.342488 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.342451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:36.411007 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.410980    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"]
Sep 16 16:44:36.413523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.413507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.421000 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.420969    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zbjqr\""
Sep 16 16:44:36.421000 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.420980    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 16:44:36.421926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.421903    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 16:44:36.436788 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.436742    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"]
Sep 16 16:44:36.452292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.452269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtbb4\" (UniqueName: \"kubernetes.io/projected/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-kube-api-access-gtbb4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.452376 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.452318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.452376 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.452341    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.552756 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.552722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.552926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.552763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.552926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.552835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtbb4\" (UniqueName: \"kubernetes.io/projected/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-kube-api-access-gtbb4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.553176 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.553154    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.553256 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.553185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.579924 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.579904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtbb4\" (UniqueName: \"kubernetes.io/projected/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-kube-api-access-gtbb4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:36.721757 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.721685    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:37.000003 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:36.999929    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"]
Sep 16 16:44:37.003021 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:44:37.002993    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19022bdf_f0b4_4ac2_b7fb_4665a92222c0.slice/crio-9f1415a25e496ed4e85d4477875ffdc52c1a26641b44751ab74fb064137176b6 WatchSource:0}: Error finding container 9f1415a25e496ed4e85d4477875ffdc52c1a26641b44751ab74fb064137176b6: Status 404 returned error can't find the container with id 9f1415a25e496ed4e85d4477875ffdc52c1a26641b44751ab74fb064137176b6
Sep 16 16:44:37.765620 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:37.765586    2577 generic.go:356] "Generic (PLEG): container finished" podID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerID="d8862ab61e8b2ca26a6e507030079e44e950193545129427f520da6f414936b2" exitCode=0
Sep 16 16:44:37.765780 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:37.765669    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8" event={"ID":"19022bdf-f0b4-4ac2-b7fb-4665a92222c0","Type":"ContainerDied","Data":"d8862ab61e8b2ca26a6e507030079e44e950193545129427f520da6f414936b2"}
Sep 16 16:44:37.765780 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:37.765713    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8" event={"ID":"19022bdf-f0b4-4ac2-b7fb-4665a92222c0","Type":"ContainerStarted","Data":"9f1415a25e496ed4e85d4477875ffdc52c1a26641b44751ab74fb064137176b6"}
Sep 16 16:44:37.966000 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:37.965972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:38.661245 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:38.661208    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:40.696730 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:40.696692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:40.781948 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:40.781912    2577 generic.go:356] "Generic (PLEG): container finished" podID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerID="c3d5d2c888d5984eab40eed0eb7b3052a0f10491010acfff264dab239c5b7b9f" exitCode=0
Sep 16 16:44:40.782106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:40.781975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8" event={"ID":"19022bdf-f0b4-4ac2-b7fb-4665a92222c0","Type":"ContainerDied","Data":"c3d5d2c888d5984eab40eed0eb7b3052a0f10491010acfff264dab239c5b7b9f"}
Sep 16 16:44:40.793361 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:40.793334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:41.788426 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:41.788392    2577 generic.go:356] "Generic (PLEG): container finished" podID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerID="3cc3254af7f0e41dc7b14822b283b9b2abd5c5370286e607f3060ea78840c4cd" exitCode=0
Sep 16 16:44:41.788819 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:41.788461    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8" event={"ID":"19022bdf-f0b4-4ac2-b7fb-4665a92222c0","Type":"ContainerDied","Data":"3cc3254af7f0e41dc7b14822b283b9b2abd5c5370286e607f3060ea78840c4cd"}
Sep 16 16:44:42.910172 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:42.910150    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:42.994008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:42.993985    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtbb4\" (UniqueName: \"kubernetes.io/projected/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-kube-api-access-gtbb4\") pod \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") "
Sep 16 16:44:42.994138 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:42.994041    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-util\") pod \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") "
Sep 16 16:44:42.994138 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:42.994063    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-bundle\") pod \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\" (UID: \"19022bdf-f0b4-4ac2-b7fb-4665a92222c0\") "
Sep 16 16:44:42.994443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:42.994418    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-bundle" (OuterVolumeSpecName: "bundle") pod "19022bdf-f0b4-4ac2-b7fb-4665a92222c0" (UID: "19022bdf-f0b4-4ac2-b7fb-4665a92222c0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:42.996066 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:42.996044    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-kube-api-access-gtbb4" (OuterVolumeSpecName: "kube-api-access-gtbb4") pod "19022bdf-f0b4-4ac2-b7fb-4665a92222c0" (UID: "19022bdf-f0b4-4ac2-b7fb-4665a92222c0"). InnerVolumeSpecName "kube-api-access-gtbb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:44:42.998507 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:42.998478    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-util" (OuterVolumeSpecName: "util") pod "19022bdf-f0b4-4ac2-b7fb-4665a92222c0" (UID: "19022bdf-f0b4-4ac2-b7fb-4665a92222c0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:44:43.094629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:43.094563    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtbb4\" (UniqueName: \"kubernetes.io/projected/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-kube-api-access-gtbb4\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:44:43.094629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:43.094589    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-util\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:44:43.094629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:43.094599    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/19022bdf-f0b4-4ac2-b7fb-4665a92222c0-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:44:43.799565 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:43.799513    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8" event={"ID":"19022bdf-f0b4-4ac2-b7fb-4665a92222c0","Type":"ContainerDied","Data":"9f1415a25e496ed4e85d4477875ffdc52c1a26641b44751ab74fb064137176b6"}
Sep 16 16:44:43.799729 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:43.799580    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f1415a25e496ed4e85d4477875ffdc52c1a26641b44751ab74fb064137176b6"
Sep 16 16:44:43.799729 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:43.799544    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd56v8"
Sep 16 16:44:44.339698 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:44.339662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:44.439595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:44.439563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:46.336482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:46.336449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:48.787716 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:48.787686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:48.886236 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:48.886203    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:50.147259 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.147222    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"]
Sep 16 16:44:50.147634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.147552    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerName="extract"
Sep 16 16:44:50.147634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.147563    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerName="extract"
Sep 16 16:44:50.147634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.147573    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerName="util"
Sep 16 16:44:50.147634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.147580    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerName="util"
Sep 16 16:44:50.147634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.147593    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerName="pull"
Sep 16 16:44:50.147634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.147598    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerName="pull"
Sep 16 16:44:50.147835 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.147647    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="19022bdf-f0b4-4ac2-b7fb-4665a92222c0" containerName="extract"
Sep 16 16:44:50.180227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.180198    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"]
Sep 16 16:44:50.180345 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.180292    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"
Sep 16 16:44:50.185114 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.185083    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 16:44:50.185114 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.185102    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 16:44:50.186077 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.186055    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-t9v9h\""
Sep 16 16:44:50.247750 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.247719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e48a-7574-4be7-9c6f-18addad37c36-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r2clv\" (UID: \"9ef8e48a-7574-4be7-9c6f-18addad37c36\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"
Sep 16 16:44:50.247941 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.247766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zht5l\" (UniqueName: \"kubernetes.io/projected/9ef8e48a-7574-4be7-9c6f-18addad37c36-kube-api-access-zht5l\") pod \"openshift-lws-operator-fd8ccff4c-r2clv\" (UID: \"9ef8e48a-7574-4be7-9c6f-18addad37c36\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"
Sep 16 16:44:50.348605 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.348576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e48a-7574-4be7-9c6f-18addad37c36-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r2clv\" (UID: \"9ef8e48a-7574-4be7-9c6f-18addad37c36\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"
Sep 16 16:44:50.348738 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.348612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zht5l\" (UniqueName: \"kubernetes.io/projected/9ef8e48a-7574-4be7-9c6f-18addad37c36-kube-api-access-zht5l\") pod \"openshift-lws-operator-fd8ccff4c-r2clv\" (UID: \"9ef8e48a-7574-4be7-9c6f-18addad37c36\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"
Sep 16 16:44:50.348980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.348958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ef8e48a-7574-4be7-9c6f-18addad37c36-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r2clv\" (UID: \"9ef8e48a-7574-4be7-9c6f-18addad37c36\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"
Sep 16 16:44:50.357366 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.357342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zht5l\" (UniqueName: \"kubernetes.io/projected/9ef8e48a-7574-4be7-9c6f-18addad37c36-kube-api-access-zht5l\") pod \"openshift-lws-operator-fd8ccff4c-r2clv\" (UID: \"9ef8e48a-7574-4be7-9c6f-18addad37c36\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"
Sep 16 16:44:50.496683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.496609    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"
Sep 16 16:44:50.783829 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.783746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv"]
Sep 16 16:44:50.786285 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:44:50.786253    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ef8e48a_7574_4be7_9c6f_18addad37c36.slice/crio-ceed2f93ed5aded82d4e6d3cb6ec26662b4fa4e9ab8979f82458be1b23d89e95 WatchSource:0}: Error finding container ceed2f93ed5aded82d4e6d3cb6ec26662b4fa4e9ab8979f82458be1b23d89e95: Status 404 returned error can't find the container with id ceed2f93ed5aded82d4e6d3cb6ec26662b4fa4e9ab8979f82458be1b23d89e95
Sep 16 16:44:50.830489 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:50.830458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv" event={"ID":"9ef8e48a-7574-4be7-9c6f-18addad37c36","Type":"ContainerStarted","Data":"ceed2f93ed5aded82d4e6d3cb6ec26662b4fa4e9ab8979f82458be1b23d89e95"}
Sep 16 16:44:53.846552 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:53.846485    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv" event={"ID":"9ef8e48a-7574-4be7-9c6f-18addad37c36","Type":"ContainerStarted","Data":"7f034cb78ee36c2b404bca6016b40f96ddc294dffbf61147c5f365d7706f08b6"}
Sep 16 16:44:53.863050 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:53.863008    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r2clv" podStartSLOduration=1.5534276550000001 podStartE2EDuration="3.862992874s" totalImagesPullingTime="2.309565219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:44:53.862166514 +0000 UTC m=+509.359039638" watchObservedRunningTime="2026-09-16 16:44:53.862992874 +0000 UTC m=+509.359865997"
Sep 16 16:44:54.450692 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:54.450660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:54.550971 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:54.550934    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:44:56.162262 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:44:56.162227    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:10.298494 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.298455    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"]
Sep 16 16:45:10.300783 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.300766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.303649 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.303627    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 16:45:10.304646 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.304629    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 16:45:10.304735 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.304720    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zbjqr\""
Sep 16 16:45:10.312164 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.312142    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"]
Sep 16 16:45:10.413958 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.413929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.414074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.413981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ftj4\" (UniqueName: \"kubernetes.io/projected/e39027af-51f9-47c3-b6a3-5ae518bf548c-kube-api-access-5ftj4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.414074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.414032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.515199 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.515170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.515362 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.515221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ftj4\" (UniqueName: \"kubernetes.io/projected/e39027af-51f9-47c3-b6a3-5ae518bf548c-kube-api-access-5ftj4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.515362 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.515250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.515566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.515546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.515611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.515572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.524788 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.524741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ftj4\" (UniqueName: \"kubernetes.io/projected/e39027af-51f9-47c3-b6a3-5ae518bf548c-kube-api-access-5ftj4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.610590 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.610520    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:10.882134 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.882114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"]
Sep 16 16:45:10.884436 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:45:10.884401    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39027af_51f9_47c3_b6a3_5ae518bf548c.slice/crio-11d86c94180bdbcb3a9fda5dc4ff405ddb3130282758c05adde6cd1275b0a1f9 WatchSource:0}: Error finding container 11d86c94180bdbcb3a9fda5dc4ff405ddb3130282758c05adde6cd1275b0a1f9: Status 404 returned error can't find the container with id 11d86c94180bdbcb3a9fda5dc4ff405ddb3130282758c05adde6cd1275b0a1f9
Sep 16 16:45:10.918584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:10.918558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb" event={"ID":"e39027af-51f9-47c3-b6a3-5ae518bf548c","Type":"ContainerStarted","Data":"11d86c94180bdbcb3a9fda5dc4ff405ddb3130282758c05adde6cd1275b0a1f9"}
Sep 16 16:45:11.924163 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:11.924130    2577 generic.go:356] "Generic (PLEG): container finished" podID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerID="4a6e30cc731a85b72147421b127be69c6587d934c543180a19e4f4b193508f49" exitCode=0
Sep 16 16:45:11.924543 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:11.924200    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb" event={"ID":"e39027af-51f9-47c3-b6a3-5ae518bf548c","Type":"ContainerDied","Data":"4a6e30cc731a85b72147421b127be69c6587d934c543180a19e4f4b193508f49"}
Sep 16 16:45:12.930743 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:12.930710    2577 generic.go:356] "Generic (PLEG): container finished" podID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerID="ab4a73dd301788e3b86ae94e182b7a78e80febfe0f0e4c302c2bccc90dd5cdef" exitCode=0
Sep 16 16:45:12.931103 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:12.930785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb" event={"ID":"e39027af-51f9-47c3-b6a3-5ae518bf548c","Type":"ContainerDied","Data":"ab4a73dd301788e3b86ae94e182b7a78e80febfe0f0e4c302c2bccc90dd5cdef"}
Sep 16 16:45:13.937899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:13.937854    2577 generic.go:356] "Generic (PLEG): container finished" podID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerID="db681977cd141eeeaad3adafecc88876b476c8167c33c91a816e02af4d7705b4" exitCode=0
Sep 16 16:45:13.938274 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:13.937946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb" event={"ID":"e39027af-51f9-47c3-b6a3-5ae518bf548c","Type":"ContainerDied","Data":"db681977cd141eeeaad3adafecc88876b476c8167c33c91a816e02af4d7705b4"}
Sep 16 16:45:15.069096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.069068    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:15.157150 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.157119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ftj4\" (UniqueName: \"kubernetes.io/projected/e39027af-51f9-47c3-b6a3-5ae518bf548c-kube-api-access-5ftj4\") pod \"e39027af-51f9-47c3-b6a3-5ae518bf548c\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") "
Sep 16 16:45:15.157150 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.157153    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-util\") pod \"e39027af-51f9-47c3-b6a3-5ae518bf548c\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") "
Sep 16 16:45:15.157323 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.157244    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-bundle\") pod \"e39027af-51f9-47c3-b6a3-5ae518bf548c\" (UID: \"e39027af-51f9-47c3-b6a3-5ae518bf548c\") "
Sep 16 16:45:15.157940 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.157909    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-bundle" (OuterVolumeSpecName: "bundle") pod "e39027af-51f9-47c3-b6a3-5ae518bf548c" (UID: "e39027af-51f9-47c3-b6a3-5ae518bf548c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:45:15.159299 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.159263    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e39027af-51f9-47c3-b6a3-5ae518bf548c-kube-api-access-5ftj4" (OuterVolumeSpecName: "kube-api-access-5ftj4") pod "e39027af-51f9-47c3-b6a3-5ae518bf548c" (UID: "e39027af-51f9-47c3-b6a3-5ae518bf548c"). InnerVolumeSpecName "kube-api-access-5ftj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:45:15.162286 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.162261    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-util" (OuterVolumeSpecName: "util") pod "e39027af-51f9-47c3-b6a3-5ae518bf548c" (UID: "e39027af-51f9-47c3-b6a3-5ae518bf548c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:45:15.257993 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.257907    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:45:15.257993 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.257945    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ftj4\" (UniqueName: \"kubernetes.io/projected/e39027af-51f9-47c3-b6a3-5ae518bf548c-kube-api-access-5ftj4\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:45:15.257993 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.257955    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e39027af-51f9-47c3-b6a3-5ae518bf548c-util\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:45:15.948595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.948567    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb"
Sep 16 16:45:15.948595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.948574    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdp2hb" event={"ID":"e39027af-51f9-47c3-b6a3-5ae518bf548c","Type":"ContainerDied","Data":"11d86c94180bdbcb3a9fda5dc4ff405ddb3130282758c05adde6cd1275b0a1f9"}
Sep 16 16:45:15.948794 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:15.948619    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11d86c94180bdbcb3a9fda5dc4ff405ddb3130282758c05adde6cd1275b0a1f9"
Sep 16 16:45:25.168669 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.168596    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"]
Sep 16 16:45:25.169074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.168875    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerName="util"
Sep 16 16:45:25.169074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.168886    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerName="util"
Sep 16 16:45:25.169074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.168898    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerName="extract"
Sep 16 16:45:25.169074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.168903    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerName="extract"
Sep 16 16:45:25.169074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.168921    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerName="pull"
Sep 16 16:45:25.169074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.168926    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerName="pull"
Sep 16 16:45:25.169074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.168980    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e39027af-51f9-47c3-b6a3-5ae518bf548c" containerName="extract"
Sep 16 16:45:25.171918 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.171901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.174831 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.174808    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 16:45:25.175109 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.175089    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zbjqr\""
Sep 16 16:45:25.175200 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.175167    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 16:45:25.188289 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.188267    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"]
Sep 16 16:45:25.339346 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.339311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.339346 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.339359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.339586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.339379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27kfr\" (UniqueName: \"kubernetes.io/projected/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-kube-api-access-27kfr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.440795 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.440713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.440795 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.440760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.440795 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.440786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27kfr\" (UniqueName: \"kubernetes.io/projected/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-kube-api-access-27kfr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.441106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.441086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.441169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.441152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.450034 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.450013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27kfr\" (UniqueName: \"kubernetes.io/projected/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-kube-api-access-27kfr\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.481111 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.481082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:25.753812 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.753726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"]
Sep 16 16:45:25.756773 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:45:25.756738    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1ff71de_4a7a_40ef_99f5_b32bba1d894c.slice/crio-50427b107351619a602d4fba226f85fb41cfcf4cfc1886815c598dd05a25fc80 WatchSource:0}: Error finding container 50427b107351619a602d4fba226f85fb41cfcf4cfc1886815c598dd05a25fc80: Status 404 returned error can't find the container with id 50427b107351619a602d4fba226f85fb41cfcf4cfc1886815c598dd05a25fc80
Sep 16 16:45:25.992839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.992806    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerID="8663c4ff857c0fece2c8c3b8ceaa19a7525fabc37523c987dbe365240ae6dc28" exitCode=0
Sep 16 16:45:25.992839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.992842    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v" event={"ID":"c1ff71de-4a7a-40ef-99f5-b32bba1d894c","Type":"ContainerDied","Data":"8663c4ff857c0fece2c8c3b8ceaa19a7525fabc37523c987dbe365240ae6dc28"}
Sep 16 16:45:25.993049 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:25.992879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v" event={"ID":"c1ff71de-4a7a-40ef-99f5-b32bba1d894c","Type":"ContainerStarted","Data":"50427b107351619a602d4fba226f85fb41cfcf4cfc1886815c598dd05a25fc80"}
Sep 16 16:45:26.534645 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:26.534610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:26.640541 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:26.640493    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:26.999061 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:26.998980    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerID="08cd380fc22015d78a8f153c93cc8b72b0cc2e6cb16dc530f2fc4334ffd794e6" exitCode=0
Sep 16 16:45:26.999061 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:26.999038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v" event={"ID":"c1ff71de-4a7a-40ef-99f5-b32bba1d894c","Type":"ContainerDied","Data":"08cd380fc22015d78a8f153c93cc8b72b0cc2e6cb16dc530f2fc4334ffd794e6"}
Sep 16 16:45:27.125766 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.125731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:27.422080 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.422049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:27.530500 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.530470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:27.582566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.582519    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"]
Sep 16 16:45:27.587232 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.587208    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.589667 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.589645    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 16:45:27.589810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.589791    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 16:45:27.590316 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.590302    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mhjfp\""
Sep 16 16:45:27.590522 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.590506    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 16:45:27.590648 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.590625    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 16:45:27.605050 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.605029    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"]
Sep 16 16:45:27.763595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.763488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/109611c5-d1b9-4d7f-bfa5-330861e99a0b-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.763595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.763543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/109611c5-d1b9-4d7f-bfa5-330861e99a0b-tmp\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.763779 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.763619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/109611c5-d1b9-4d7f-bfa5-330861e99a0b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.763779 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.763658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/109611c5-d1b9-4d7f-bfa5-330861e99a0b-manifests\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.763779 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.763685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zsrf\" (UniqueName: \"kubernetes.io/projected/109611c5-d1b9-4d7f-bfa5-330861e99a0b-kube-api-access-8zsrf\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.794776 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.794743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:27.864687 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.864664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/109611c5-d1b9-4d7f-bfa5-330861e99a0b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.864832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.864704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/109611c5-d1b9-4d7f-bfa5-330861e99a0b-manifests\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.864832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.864728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zsrf\" (UniqueName: \"kubernetes.io/projected/109611c5-d1b9-4d7f-bfa5-330861e99a0b-kube-api-access-8zsrf\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.864832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.864753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/109611c5-d1b9-4d7f-bfa5-330861e99a0b-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.864832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.864771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/109611c5-d1b9-4d7f-bfa5-330861e99a0b-tmp\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.865211 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.865187    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/109611c5-d1b9-4d7f-bfa5-330861e99a0b-manifests\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.867062 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.867031    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/109611c5-d1b9-4d7f-bfa5-330861e99a0b-tmp\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.867170 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.867117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/109611c5-d1b9-4d7f-bfa5-330861e99a0b-webhook-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.867312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.867293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/109611c5-d1b9-4d7f-bfa5-330861e99a0b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.872738 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.872718    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zsrf\" (UniqueName: \"kubernetes.io/projected/109611c5-d1b9-4d7f-bfa5-330861e99a0b-kube-api-access-8zsrf\") pod \"opendatahub-operator-controller-manager-54d7c7f658-g9fdp\" (UID: \"109611c5-d1b9-4d7f-bfa5-330861e99a0b\") " pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.895297 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.895275    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:27.897220 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.897202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:27.994737 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:27.994701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:28.005875 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:28.005852    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerID="fd3142573c91e00196d1d95a22b0e8399a889bf60818939f86dda943e9f39c6d" exitCode=0
Sep 16 16:45:28.005990 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:28.005919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v" event={"ID":"c1ff71de-4a7a-40ef-99f5-b32bba1d894c","Type":"ContainerDied","Data":"fd3142573c91e00196d1d95a22b0e8399a889bf60818939f86dda943e9f39c6d"}
Sep 16 16:45:28.098327 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:28.098267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:28.178567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:28.178433    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"]
Sep 16 16:45:28.181598 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:45:28.181566    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109611c5_d1b9_4d7f_bfa5_330861e99a0b.slice/crio-3a9c55b1436436ce0ffd2dfef4d59c911de0dcfc089c4b98ac431835e59c1f14 WatchSource:0}: Error finding container 3a9c55b1436436ce0ffd2dfef4d59c911de0dcfc089c4b98ac431835e59c1f14: Status 404 returned error can't find the container with id 3a9c55b1436436ce0ffd2dfef4d59c911de0dcfc089c4b98ac431835e59c1f14
Sep 16 16:45:28.183927 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:28.183896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:28.265296 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:28.265273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:29.014022 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.013944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp" event={"ID":"109611c5-d1b9-4d7f-bfa5-330861e99a0b","Type":"ContainerStarted","Data":"3a9c55b1436436ce0ffd2dfef4d59c911de0dcfc089c4b98ac431835e59c1f14"}
Sep 16 16:45:29.165762 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.165736    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:29.276814 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.276738    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-bundle\") pod \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") "
Sep 16 16:45:29.276814 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.276803    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-util\") pod \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") "
Sep 16 16:45:29.277022 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.276857    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27kfr\" (UniqueName: \"kubernetes.io/projected/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-kube-api-access-27kfr\") pod \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\" (UID: \"c1ff71de-4a7a-40ef-99f5-b32bba1d894c\") "
Sep 16 16:45:29.277715 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.277664    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-bundle" (OuterVolumeSpecName: "bundle") pod "c1ff71de-4a7a-40ef-99f5-b32bba1d894c" (UID: "c1ff71de-4a7a-40ef-99f5-b32bba1d894c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:45:29.280271 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.279543    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-kube-api-access-27kfr" (OuterVolumeSpecName: "kube-api-access-27kfr") pod "c1ff71de-4a7a-40ef-99f5-b32bba1d894c" (UID: "c1ff71de-4a7a-40ef-99f5-b32bba1d894c"). InnerVolumeSpecName "kube-api-access-27kfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:45:29.285216 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.285170    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-util" (OuterVolumeSpecName: "util") pod "c1ff71de-4a7a-40ef-99f5-b32bba1d894c" (UID: "c1ff71de-4a7a-40ef-99f5-b32bba1d894c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:45:29.378376 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.378341    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:45:29.378376 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.378376    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-util\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:45:29.378610 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:29.378392    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27kfr\" (UniqueName: \"kubernetes.io/projected/c1ff71de-4a7a-40ef-99f5-b32bba1d894c-kube-api-access-27kfr\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:45:30.022300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:30.022260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v" event={"ID":"c1ff71de-4a7a-40ef-99f5-b32bba1d894c","Type":"ContainerDied","Data":"50427b107351619a602d4fba226f85fb41cfcf4cfc1886815c598dd05a25fc80"}
Sep 16 16:45:30.022300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:30.022304    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b07bp9v"
Sep 16 16:45:30.022783 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:30.022322    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50427b107351619a602d4fba226f85fb41cfcf4cfc1886815c598dd05a25fc80"
Sep 16 16:45:30.970764 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:30.970725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:31.062887 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:31.062844    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:32.032962 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:32.032927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp" event={"ID":"109611c5-d1b9-4d7f-bfa5-330861e99a0b","Type":"ContainerStarted","Data":"0f85996c13844512b6608005c0cdf88108b87a69b2c5d9e4191a6640c2b583d1"}
Sep 16 16:45:33.038801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:33.038766    2577 generic.go:356] "Generic (PLEG): container finished" podID="109611c5-d1b9-4d7f-bfa5-330861e99a0b" containerID="0f85996c13844512b6608005c0cdf88108b87a69b2c5d9e4191a6640c2b583d1" exitCode=0
Sep 16 16:45:33.039228 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:33.038847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp" event={"ID":"109611c5-d1b9-4d7f-bfa5-330861e99a0b","Type":"ContainerDied","Data":"0f85996c13844512b6608005c0cdf88108b87a69b2c5d9e4191a6640c2b583d1"}
Sep 16 16:45:34.046009 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:34.045975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp" event={"ID":"109611c5-d1b9-4d7f-bfa5-330861e99a0b","Type":"ContainerStarted","Data":"3301407afb412397f53502a084374acbf05ba895b444e4413d810718d0d3313f"}
Sep 16 16:45:34.046372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:34.046071    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:34.073349 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:34.073297    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp" podStartSLOduration=2.796792199 podStartE2EDuration="7.073278861s" totalImagesPullingTime="3.213066589s" totalInitContainerRuntime="1.063420073s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:45:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:45:34.071456188 +0000 UTC m=+549.568329322" watchObservedRunningTime="2026-09-16 16:45:34.073278861 +0000 UTC m=+549.570151986"
Sep 16 16:45:45.053017 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:45.052986    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54d7c7f658-g9fdp"
Sep 16 16:45:47.042251 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:47.042211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:47.284955 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:47.284920    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:55.836571 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:55.836516    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:55.897435 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:55.897407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:57.876403 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:57.876366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:57.975880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:57.975837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:58.076428 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.076380    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:58.125513 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.125479    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"]
Sep 16 16:45:58.125835 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.125822    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerName="pull"
Sep 16 16:45:58.125877 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.125836    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerName="pull"
Sep 16 16:45:58.125877 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.125849    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerName="extract"
Sep 16 16:45:58.125877 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.125855    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerName="extract"
Sep 16 16:45:58.125877 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.125862    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerName="util"
Sep 16 16:45:58.125877 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.125867    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerName="util"
Sep 16 16:45:58.126024 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.125917    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1ff71de-4a7a-40ef-99f5-b32bba1d894c" containerName="extract"
Sep 16 16:45:58.128959 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.128906    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.131538 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.131516    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 16:45:58.131660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.131548    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 16:45:58.142233 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.142211    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"]
Sep 16 16:45:58.167873 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.167812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:58.216554 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.216497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd27ec48-9012-4f57-a80d-c3925b60624c-tmp\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.216714 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.216573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd27ec48-9012-4f57-a80d-c3925b60624c-tls-certs\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.216714 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.216601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk4qf\" (UniqueName: \"kubernetes.io/projected/dd27ec48-9012-4f57-a80d-c3925b60624c-kube-api-access-kk4qf\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.232977 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.232946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:58.317995 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.317965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd27ec48-9012-4f57-a80d-c3925b60624c-tls-certs\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.318142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.318003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk4qf\" (UniqueName: \"kubernetes.io/projected/dd27ec48-9012-4f57-a80d-c3925b60624c-kube-api-access-kk4qf\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.318142 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:45:58.318109    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 16 16:45:58.318217 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:45:58.318168    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd27ec48-9012-4f57-a80d-c3925b60624c-tls-certs podName:dd27ec48-9012-4f57-a80d-c3925b60624c nodeName:}" failed. No retries permitted until 2026-09-16 16:45:58.818152247 +0000 UTC m=+574.315025349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dd27ec48-9012-4f57-a80d-c3925b60624c-tls-certs") pod "kube-auth-proxy-b966d4c98-g7fbv" (UID: "dd27ec48-9012-4f57-a80d-c3925b60624c") : secret "kube-auth-proxy-tls" not found
Sep 16 16:45:58.318217 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.318185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd27ec48-9012-4f57-a80d-c3925b60624c-tmp\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.320326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.320300    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dd27ec48-9012-4f57-a80d-c3925b60624c-tmp\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.326717 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.326694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk4qf\" (UniqueName: \"kubernetes.io/projected/dd27ec48-9012-4f57-a80d-c3925b60624c-kube-api-access-kk4qf\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.487325 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.487292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:58.596638 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.596602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:58.821999 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.821908    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd27ec48-9012-4f57-a80d-c3925b60624c-tls-certs\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:58.824359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:58.824333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd27ec48-9012-4f57-a80d-c3925b60624c-tls-certs\") pod \"kube-auth-proxy-b966d4c98-g7fbv\" (UID: \"dd27ec48-9012-4f57-a80d-c3925b60624c\") " pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:59.038310 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.038276    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"
Sep 16 16:45:59.051957 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.051925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:59.310259 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.310227    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv"]
Sep 16 16:45:59.312081 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:45:59.312052    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd27ec48_9012_4f57_a80d_c3925b60624c.slice/crio-c9c0325425f5975b71e15154bcef23294f407108da8ade8edd1cb8c93e514703 WatchSource:0}: Error finding container c9c0325425f5975b71e15154bcef23294f407108da8ade8edd1cb8c93e514703: Status 404 returned error can't find the container with id c9c0325425f5975b71e15154bcef23294f407108da8ade8edd1cb8c93e514703
Sep 16 16:45:59.388533 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.388499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:59.398977 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.398954    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"]
Sep 16 16:45:59.403343 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.403329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.407500 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.407470    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 16:45:59.407990 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.407973    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 16:45:59.408071 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.408056    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 16:45:59.408244 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.408230    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 16:45:59.408336 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.408256    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ws4cl\""
Sep 16 16:45:59.421096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.421072    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"]
Sep 16 16:45:59.494811 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.494779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:45:59.527872 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.527835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.528009 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.527931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.528009 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.527994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.528097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.528024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1474a554-c513-4296-aaad-b2dfce1ae464-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.528097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.528040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.528097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.528061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znng2\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-kube-api-access-znng2\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.528097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.528077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.629098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.629071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.629098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.629101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1474a554-c513-4296-aaad-b2dfce1ae464-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.629279 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.629116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.629279 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.629133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znng2\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-kube-api-access-znng2\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.629279 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.629151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.629429 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.629317    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.629429 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.629398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.630092 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.630063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.631509 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.631479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1474a554-c513-4296-aaad-b2dfce1ae464-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.631663 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.631637    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.631976 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.631955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.632020 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.631967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.637372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.637352    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.637572 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.637550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znng2\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-kube-api-access-znng2\") pod \"istiod-openshift-gateway-7b7995d5cd-xhnrj\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.712674 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.712606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:45:59.990259 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:45:59.990187    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"]
Sep 16 16:45:59.992663 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:45:59.992636    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1474a554_c513_4296_aaad_b2dfce1ae464.slice/crio-29a1b620bf1e221b814b3c92115fdda6ab896d871299fbf63dc93198faf82785 WatchSource:0}: Error finding container 29a1b620bf1e221b814b3c92115fdda6ab896d871299fbf63dc93198faf82785: Status 404 returned error can't find the container with id 29a1b620bf1e221b814b3c92115fdda6ab896d871299fbf63dc93198faf82785
Sep 16 16:46:00.167431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:00.167391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv" event={"ID":"dd27ec48-9012-4f57-a80d-c3925b60624c","Type":"ContainerStarted","Data":"c9c0325425f5975b71e15154bcef23294f407108da8ade8edd1cb8c93e514703"}
Sep 16 16:46:00.168969 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:00.168941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj" event={"ID":"1474a554-c513-4296-aaad-b2dfce1ae464","Type":"ContainerStarted","Data":"29a1b620bf1e221b814b3c92115fdda6ab896d871299fbf63dc93198faf82785"}
Sep 16 16:46:00.490098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:00.490061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:02.184290 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:02.183901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv" event={"ID":"dd27ec48-9012-4f57-a80d-c3925b60624c","Type":"ContainerStarted","Data":"1670ae0e6d2ddf7645ce8bd5403e74e1c55fbf9282b5f76005f2ac24c1c87f9c"}
Sep 16 16:46:02.202255 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:02.202113    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b966d4c98-g7fbv" podStartSLOduration=1.443028003 podStartE2EDuration="4.202091003s" totalImagesPullingTime="2.759063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:45:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:46:02.200699652 +0000 UTC m=+577.697572779" watchObservedRunningTime="2026-09-16 16:46:02.202091003 +0000 UTC m=+577.698964128"
Sep 16 16:46:03.826114 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:03.826075    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 16:46:03.826462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:03.826156    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 16:46:04.196853 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:04.196813    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj" event={"ID":"1474a554-c513-4296-aaad-b2dfce1ae464","Type":"ContainerStarted","Data":"aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e"}
Sep 16 16:46:04.197037 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:04.197011    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:46:04.198830 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:04.198804    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-xhnrj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 16:46:04.198939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:04.198856    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj" podUID="1474a554-c513-4296-aaad-b2dfce1ae464" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:46:04.218033 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:04.217989    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj" podStartSLOduration=1.387509057 podStartE2EDuration="5.217975454s" totalImagesPullingTime="3.830466397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:45:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:46:04.216468151 +0000 UTC m=+579.713341266" watchObservedRunningTime="2026-09-16 16:46:04.217975454 +0000 UTC m=+579.714848579"
Sep 16 16:46:05.201746 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:05.201716    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-xhnrj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 16:46:05.202114 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:05.201779    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj" podUID="1474a554-c513-4296-aaad-b2dfce1ae464" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 16:46:05.248753 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:05.248670    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:05.342417 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:05.342383    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:06.384725 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:06.384683    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:06.480856 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:06.480819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:06.581080 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:06.581047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:08.185992 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.185957    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"]
Sep 16 16:46:08.189572 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.189555    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.193244 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.193219    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 16:46:08.193244 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.193237    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 16:46:08.193440 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.193272    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-52z2s\""
Sep 16 16:46:08.193440 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.193273    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 16:46:08.201773 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.201755    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:46:08.206262 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.206241    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"]
Sep 16 16:46:08.310909 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.310875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-556tx\" (UniqueName: \"kubernetes.io/projected/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-kube-api-access-556tx\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.311091 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.310921    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-manager-config\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.311160 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.311128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-metrics-cert\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.311308 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.311283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-cert\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.411732 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.411699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-metrics-cert\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.411895 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.411750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-cert\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.411895 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.411866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-556tx\" (UniqueName: \"kubernetes.io/projected/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-kube-api-access-556tx\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.411980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.411916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-manager-config\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.412554 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.412511    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-manager-config\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.414197 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.414176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-cert\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.414295 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.414233    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-metrics-cert\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.425629 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.425605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-556tx\" (UniqueName: \"kubernetes.io/projected/b84afc05-e3a7-4031-9277-1ed5e9b6a7dc-kube-api-access-556tx\") pod \"lws-controller-manager-c5d8fb6c6-gh4q6\" (UID: \"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc\") " pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.498704 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.498638    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:08.789506 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:08.789394    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"]
Sep 16 16:46:08.792046 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:46:08.792018    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84afc05_e3a7_4031_9277_1ed5e9b6a7dc.slice/crio-bfd7bcca1a39b47f2f005d25045aa9d5b1e3018ce3f0e5f6e5e9d7ad4cdcca88 WatchSource:0}: Error finding container bfd7bcca1a39b47f2f005d25045aa9d5b1e3018ce3f0e5f6e5e9d7ad4cdcca88: Status 404 returned error can't find the container with id bfd7bcca1a39b47f2f005d25045aa9d5b1e3018ce3f0e5f6e5e9d7ad4cdcca88
Sep 16 16:46:09.222721 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:09.222684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6" event={"ID":"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc","Type":"ContainerStarted","Data":"bfd7bcca1a39b47f2f005d25045aa9d5b1e3018ce3f0e5f6e5e9d7ad4cdcca88"}
Sep 16 16:46:09.302186 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:09.302150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:09.427495 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:09.427449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:09.658301 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:09.658261    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:09.781585 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:09.781545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:10.710431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:10.710377    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:10.844635 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:10.844522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:10.899002 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:10.898968    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"]
Sep 16 16:46:10.899182 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:10.899162    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj" podUID="1474a554-c513-4296-aaad-b2dfce1ae464" containerName="discovery" containerID="cri-o://aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e" gracePeriod=30
Sep 16 16:46:11.136350 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.136328    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:46:11.234369 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.234335    2577 generic.go:356] "Generic (PLEG): container finished" podID="1474a554-c513-4296-aaad-b2dfce1ae464" containerID="aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e" exitCode=0
Sep 16 16:46:11.234557 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.234387    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"
Sep 16 16:46:11.234557 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.234419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj" event={"ID":"1474a554-c513-4296-aaad-b2dfce1ae464","Type":"ContainerDied","Data":"aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e"}
Sep 16 16:46:11.234557 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.234471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj" event={"ID":"1474a554-c513-4296-aaad-b2dfce1ae464","Type":"ContainerDied","Data":"29a1b620bf1e221b814b3c92115fdda6ab896d871299fbf63dc93198faf82785"}
Sep 16 16:46:11.234557 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.234498    2577 scope.go:117] "RemoveContainer" containerID="aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e"
Sep 16 16:46:11.236101 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.236079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6" event={"ID":"b84afc05-e3a7-4031-9277-1ed5e9b6a7dc","Type":"ContainerStarted","Data":"22c826baa3cecb7ca1db50d68f90ed2df55e40d4c38e06481bfe65c45282a50a"}
Sep 16 16:46:11.236239 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.236221    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:11.240639 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.240616    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-kubeconfig\") pod \"1474a554-c513-4296-aaad-b2dfce1ae464\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") "
Sep 16 16:46:11.240730 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.240714    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-cacerts\") pod \"1474a554-c513-4296-aaad-b2dfce1ae464\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") "
Sep 16 16:46:11.240786 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.240762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-ca-configmap\") pod \"1474a554-c513-4296-aaad-b2dfce1ae464\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") "
Sep 16 16:46:11.240839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.240785    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-dns-cert\") pod \"1474a554-c513-4296-aaad-b2dfce1ae464\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") "
Sep 16 16:46:11.240904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.240846    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-istio-token\") pod \"1474a554-c513-4296-aaad-b2dfce1ae464\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") "
Sep 16 16:46:11.240904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.240886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1474a554-c513-4296-aaad-b2dfce1ae464-local-certs\") pod \"1474a554-c513-4296-aaad-b2dfce1ae464\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") "
Sep 16 16:46:11.241006 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.240934    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znng2\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-kube-api-access-znng2\") pod \"1474a554-c513-4296-aaad-b2dfce1ae464\" (UID: \"1474a554-c513-4296-aaad-b2dfce1ae464\") "
Sep 16 16:46:11.241433 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.241402    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1474a554-c513-4296-aaad-b2dfce1ae464" (UID: "1474a554-c513-4296-aaad-b2dfce1ae464"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:46:11.243261 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.243227    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1474a554-c513-4296-aaad-b2dfce1ae464-local-certs" (OuterVolumeSpecName: "local-certs") pod "1474a554-c513-4296-aaad-b2dfce1ae464" (UID: "1474a554-c513-4296-aaad-b2dfce1ae464"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:46:11.243369 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.243310    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-cacerts" (OuterVolumeSpecName: "cacerts") pod "1474a554-c513-4296-aaad-b2dfce1ae464" (UID: "1474a554-c513-4296-aaad-b2dfce1ae464"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:46:11.243369 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.243321    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1474a554-c513-4296-aaad-b2dfce1ae464" (UID: "1474a554-c513-4296-aaad-b2dfce1ae464"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:46:11.243466 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.243452    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-kube-api-access-znng2" (OuterVolumeSpecName: "kube-api-access-znng2") pod "1474a554-c513-4296-aaad-b2dfce1ae464" (UID: "1474a554-c513-4296-aaad-b2dfce1ae464"). InnerVolumeSpecName "kube-api-access-znng2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:46:11.243645 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.243612    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1474a554-c513-4296-aaad-b2dfce1ae464" (UID: "1474a554-c513-4296-aaad-b2dfce1ae464"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:46:11.243740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.243685    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-istio-token" (OuterVolumeSpecName: "istio-token") pod "1474a554-c513-4296-aaad-b2dfce1ae464" (UID: "1474a554-c513-4296-aaad-b2dfce1ae464"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:46:11.244274 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.244255    2577 scope.go:117] "RemoveContainer" containerID="aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e"
Sep 16 16:46:11.244578 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:46:11.244561    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e\": container with ID starting with aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e not found: ID does not exist" containerID="aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e"
Sep 16 16:46:11.244634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.244587    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e"} err="failed to get container status \"aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e\": rpc error: code = NotFound desc = could not find container \"aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e\": container with ID starting with aac39d5904e0d3cf855c585b8031cafd14e5744e4348ef5d3ca9c0d63b2b571e not found: ID does not exist"
Sep 16 16:46:11.266523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.266479    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6" podStartSLOduration=1.511287386 podStartE2EDuration="3.266465986s" totalImagesPullingTime="1.7551786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:46:11.264609907 +0000 UTC m=+586.761483034" watchObservedRunningTime="2026-09-16 16:46:11.266465986 +0000 UTC m=+586.763339110"
Sep 16 16:46:11.342395 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.342361    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-kubeconfig\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:46:11.342395 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.342393    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-cacerts\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:46:11.342640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.342407    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-ca-configmap\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:46:11.342640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.342419    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1474a554-c513-4296-aaad-b2dfce1ae464-istio-csr-dns-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:46:11.342640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.342432    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-istio-token\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:46:11.342640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.342444    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1474a554-c513-4296-aaad-b2dfce1ae464-local-certs\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:46:11.342640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.342457    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znng2\" (UniqueName: \"kubernetes.io/projected/1474a554-c513-4296-aaad-b2dfce1ae464-kube-api-access-znng2\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:46:11.556827 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.556801    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"]
Sep 16 16:46:11.564087 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:11.564062    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xhnrj"]
Sep 16 16:46:12.977164 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:12.977127    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1474a554-c513-4296-aaad-b2dfce1ae464" path="/var/lib/kubelet/pods/1474a554-c513-4296-aaad-b2dfce1ae464/volumes"
Sep 16 16:46:16.568233 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:16.568190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:22.243563 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:22.243513    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-c5d8fb6c6-gh4q6"
Sep 16 16:46:24.879771 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:24.879742    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:46:24.880248 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:24.880228    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:46:24.921414 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:24.921388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:26.186589 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:26.186547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:26.727034 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:26.727001    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:26.804142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:26.804114    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:29.723026 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:29.722990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:29.780468 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:29.780438    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:59.013475 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:59.013389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:46:59.105813 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:46:59.105779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:02.842108 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.842031    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"]
Sep 16 16:47:02.842492 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.842364    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1474a554-c513-4296-aaad-b2dfce1ae464" containerName="discovery"
Sep 16 16:47:02.842492 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.842374    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1474a554-c513-4296-aaad-b2dfce1ae464" containerName="discovery"
Sep 16 16:47:02.842492 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.842439    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1474a554-c513-4296-aaad-b2dfce1ae464" containerName="discovery"
Sep 16 16:47:02.844483 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.844467    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:02.846863 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.846839    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 16 16:47:02.846985 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.846883    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zbjqr\""
Sep 16 16:47:02.846985 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.846920    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 16 16:47:02.853654 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.853635    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"]
Sep 16 16:47:02.941215 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.941188    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"]
Sep 16 16:47:02.943411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.943396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:02.953411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.953385    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"]
Sep 16 16:47:02.953499 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.953441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpw45\" (UniqueName: \"kubernetes.io/projected/ae6a3124-b093-41c7-be5c-ae04bfbbb556-kube-api-access-cpw45\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:02.953499 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.953474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:02.953645 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:02.953572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:03.037140 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.037107    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"]
Sep 16 16:47:03.039296 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.039275    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.048481 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.048456    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"]
Sep 16 16:47:03.053955 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.053932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.054065 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.053972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpw45\" (UniqueName: \"kubernetes.io/projected/ae6a3124-b093-41c7-be5c-ae04bfbbb556-kube-api-access-cpw45\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:03.054065 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.053990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:03.054065 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.054027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrvv5\" (UniqueName: \"kubernetes.io/projected/a4660274-0926-422a-859e-d8d906c5bf07-kube-api-access-rrvv5\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.054065 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.054064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:03.054290 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.054187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.054378 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.054361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:03.054512 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.054497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:03.064625 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.064597    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpw45\" (UniqueName: \"kubernetes.io/projected/ae6a3124-b093-41c7-be5c-ae04bfbbb556-kube-api-access-cpw45\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:03.147449 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.147427    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"]
Sep 16 16:47:03.149896 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.149881    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.153657 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.153641    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:03.155686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.155229    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.155686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.155265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.155686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.155309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.155686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.155335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.155686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.155359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwz9j\" (UniqueName: \"kubernetes.io/projected/f2d0e62f-b0f5-4896-af1e-14358df91148-kube-api-access-gwz9j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.155686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.155392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrvv5\" (UniqueName: \"kubernetes.io/projected/a4660274-0926-422a-859e-d8d906c5bf07-kube-api-access-rrvv5\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.155686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.155618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.155996 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.155822    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.161352 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.161317    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"]
Sep 16 16:47:03.167950 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.167927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrvv5\" (UniqueName: \"kubernetes.io/projected/a4660274-0926-422a-859e-d8d906c5bf07-kube-api-access-rrvv5\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.252556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.252503    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:03.256168 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.256141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.256304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.256177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwz9j\" (UniqueName: \"kubernetes.io/projected/f2d0e62f-b0f5-4896-af1e-14358df91148-kube-api-access-gwz9j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.256304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.256206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.256304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.256260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.256304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.256286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.256523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.256303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt79m\" (UniqueName: \"kubernetes.io/projected/a918f3e8-6895-4353-9639-50fc7dc066cc-kube-api-access-zt79m\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.256605 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.256582    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.256665 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.256587    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.267813 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.267790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwz9j\" (UniqueName: \"kubernetes.io/projected/f2d0e62f-b0f5-4896-af1e-14358df91148-kube-api-access-gwz9j\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.348705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.348680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:03.357334 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.357306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.357406 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.357360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.357406 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.357387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt79m\" (UniqueName: \"kubernetes.io/projected/a918f3e8-6895-4353-9639-50fc7dc066cc-kube-api-access-zt79m\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.357703 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.357681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.357703 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.357695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.375275 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.375247    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt79m\" (UniqueName: \"kubernetes.io/projected/a918f3e8-6895-4353-9639-50fc7dc066cc-kube-api-access-zt79m\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.431409 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.431380    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"]
Sep 16 16:47:03.432953 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:47:03.432929    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6a3124_b093_41c7_be5c_ae04bfbbb556.slice/crio-a14ff02b50ba227a76a416785c19f69e0540fe49b6d1cd98d60ecc5978ed4627 WatchSource:0}: Error finding container a14ff02b50ba227a76a416785c19f69e0540fe49b6d1cd98d60ecc5978ed4627: Status 404 returned error can't find the container with id a14ff02b50ba227a76a416785c19f69e0540fe49b6d1cd98d60ecc5978ed4627
Sep 16 16:47:03.460000 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.459980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:03.481457 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.481425    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk" event={"ID":"ae6a3124-b093-41c7-be5c-ae04bfbbb556","Type":"ContainerStarted","Data":"a14ff02b50ba227a76a416785c19f69e0540fe49b6d1cd98d60ecc5978ed4627"}
Sep 16 16:47:03.530416 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.530382    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"]
Sep 16 16:47:03.552613 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:47:03.552589    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4660274_0926_422a_859e_d8d906c5bf07.slice/crio-b5f065c95c4c73c3068286abb223a4bf71cc1db4ad4aea4a601da547541c09fa WatchSource:0}: Error finding container b5f065c95c4c73c3068286abb223a4bf71cc1db4ad4aea4a601da547541c09fa: Status 404 returned error can't find the container with id b5f065c95c4c73c3068286abb223a4bf71cc1db4ad4aea4a601da547541c09fa
Sep 16 16:47:03.618633 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.618603    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"]
Sep 16 16:47:03.620511 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:47:03.620490    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d0e62f_b0f5_4896_af1e_14358df91148.slice/crio-0516353de9e8e6126ad10adda1c828c94ead4aea0f6303784c68c2e0b1c6cb49 WatchSource:0}: Error finding container 0516353de9e8e6126ad10adda1c828c94ead4aea0f6303784c68c2e0b1c6cb49: Status 404 returned error can't find the container with id 0516353de9e8e6126ad10adda1c828c94ead4aea0f6303784c68c2e0b1c6cb49
Sep 16 16:47:03.734339 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.734312    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"]
Sep 16 16:47:03.775359 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:47:03.775332    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda918f3e8_6895_4353_9639_50fc7dc066cc.slice/crio-6f829274fedc07ed1f301c031dbe5bacf1f69bf0e041a1fb5a20bb531e6837c2 WatchSource:0}: Error finding container 6f829274fedc07ed1f301c031dbe5bacf1f69bf0e041a1fb5a20bb531e6837c2: Status 404 returned error can't find the container with id 6f829274fedc07ed1f301c031dbe5bacf1f69bf0e041a1fb5a20bb531e6837c2
Sep 16 16:47:03.907477 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.907095    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cc5664dbc-8dnf5"]
Sep 16 16:47:03.909823 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.909803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:03.921070 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.921044    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cc5664dbc-8dnf5"]
Sep 16 16:47:03.962601 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.962574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-oauth-config\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:03.962710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.962619    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-config\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:03.962710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.962647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-trusted-ca-bundle\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:03.962710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.962669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-oauth-serving-cert\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:03.962710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.962691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqvsz\" (UniqueName: \"kubernetes.io/projected/24eeec43-4bf5-4edd-816c-4f2b351eb077-kube-api-access-nqvsz\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:03.962710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.962707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-serving-cert\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:03.962957 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.962816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-service-ca\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:03.971461 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:03.971389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:04.035388 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.035366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:04.063945 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.063920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-oauth-config\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.064058 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.063956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-config\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.064058 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.063974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-trusted-ca-bundle\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.064058 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.063988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-oauth-serving-cert\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.064277 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.064258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqvsz\" (UniqueName: \"kubernetes.io/projected/24eeec43-4bf5-4edd-816c-4f2b351eb077-kube-api-access-nqvsz\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.064361 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.064298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-serving-cert\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.064417 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.064371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-service-ca\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.064755 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.064729    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-oauth-serving-cert\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.064910 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.064826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-config\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.065010 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.064989    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-service-ca\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.065239 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.065218    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24eeec43-4bf5-4edd-816c-4f2b351eb077-trusted-ca-bundle\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.066550 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.066517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-oauth-config\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.067130 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.067114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24eeec43-4bf5-4edd-816c-4f2b351eb077-console-serving-cert\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.072743 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.072726    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqvsz\" (UniqueName: \"kubernetes.io/projected/24eeec43-4bf5-4edd-816c-4f2b351eb077-kube-api-access-nqvsz\") pod \"console-5cc5664dbc-8dnf5\" (UID: \"24eeec43-4bf5-4edd-816c-4f2b351eb077\") " pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.234765 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.234669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:04.488570 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.488478    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerID="fb957a13f7559dfe82d4fd884cb84b2341e5f457ae0afd6eda6ad02768e3ed3e" exitCode=0
Sep 16 16:47:04.488570 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.488559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk" event={"ID":"ae6a3124-b093-41c7-be5c-ae04bfbbb556","Type":"ContainerDied","Data":"fb957a13f7559dfe82d4fd884cb84b2341e5f457ae0afd6eda6ad02768e3ed3e"}
Sep 16 16:47:04.489665 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.489645    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:47:04.490209 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.490122    2577 generic.go:356] "Generic (PLEG): container finished" podID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerID="40c571eaddc6bce43936fcba2f5cd1c0240cfcbfa337601cff30f4e98e9bc6c1" exitCode=0
Sep 16 16:47:04.490260 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.490208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw" event={"ID":"a918f3e8-6895-4353-9639-50fc7dc066cc","Type":"ContainerDied","Data":"40c571eaddc6bce43936fcba2f5cd1c0240cfcbfa337601cff30f4e98e9bc6c1"}
Sep 16 16:47:04.490309 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.490257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw" event={"ID":"a918f3e8-6895-4353-9639-50fc7dc066cc","Type":"ContainerStarted","Data":"6f829274fedc07ed1f301c031dbe5bacf1f69bf0e041a1fb5a20bb531e6837c2"}
Sep 16 16:47:04.492069 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.492048    2577 generic.go:356] "Generic (PLEG): container finished" podID="a4660274-0926-422a-859e-d8d906c5bf07" containerID="43369721fffbe4d368f893b575ed9ac9929c27f95fdcd22316f15c032632815e" exitCode=0
Sep 16 16:47:04.492170 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.492124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8" event={"ID":"a4660274-0926-422a-859e-d8d906c5bf07","Type":"ContainerDied","Data":"43369721fffbe4d368f893b575ed9ac9929c27f95fdcd22316f15c032632815e"}
Sep 16 16:47:04.492170 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.492162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8" event={"ID":"a4660274-0926-422a-859e-d8d906c5bf07","Type":"ContainerStarted","Data":"b5f065c95c4c73c3068286abb223a4bf71cc1db4ad4aea4a601da547541c09fa"}
Sep 16 16:47:04.494004 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.493987    2577 generic.go:356] "Generic (PLEG): container finished" podID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerID="66eb503d04f50bb4e732a4a14585fc2a1906cf5b6d177f91262e8d456d54c492" exitCode=0
Sep 16 16:47:04.494098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.494043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79" event={"ID":"f2d0e62f-b0f5-4896-af1e-14358df91148","Type":"ContainerDied","Data":"66eb503d04f50bb4e732a4a14585fc2a1906cf5b6d177f91262e8d456d54c492"}
Sep 16 16:47:04.494098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.494088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79" event={"ID":"f2d0e62f-b0f5-4896-af1e-14358df91148","Type":"ContainerStarted","Data":"0516353de9e8e6126ad10adda1c828c94ead4aea0f6303784c68c2e0b1c6cb49"}
Sep 16 16:47:04.507347 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:04.507286    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cc5664dbc-8dnf5"]
Sep 16 16:47:04.509842 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:47:04.509819    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24eeec43_4bf5_4edd_816c_4f2b351eb077.slice/crio-00bf1f56130e7f92b9270ac5fb060fc519470a5b642e55c6e850e8daca5a9db3 WatchSource:0}: Error finding container 00bf1f56130e7f92b9270ac5fb060fc519470a5b642e55c6e850e8daca5a9db3: Status 404 returned error can't find the container with id 00bf1f56130e7f92b9270ac5fb060fc519470a5b642e55c6e850e8daca5a9db3
Sep 16 16:47:05.504199 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:05.504173    2577 generic.go:356] "Generic (PLEG): container finished" podID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerID="e1992a0922209e0bdbbdf5058cf6625f7b7cb5cd03c204fe7a06b9695e6a275d" exitCode=0
Sep 16 16:47:05.504592 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:05.504295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79" event={"ID":"f2d0e62f-b0f5-4896-af1e-14358df91148","Type":"ContainerDied","Data":"e1992a0922209e0bdbbdf5058cf6625f7b7cb5cd03c204fe7a06b9695e6a275d"}
Sep 16 16:47:05.507106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:05.507081    2577 generic.go:356] "Generic (PLEG): container finished" podID="a4660274-0926-422a-859e-d8d906c5bf07" containerID="1c2dcfd2d2b359ed2e869200e415689acd5fb5191723f7e332b7aa18e3ad74ad" exitCode=0
Sep 16 16:47:05.507192 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:05.507168    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8" event={"ID":"a4660274-0926-422a-859e-d8d906c5bf07","Type":"ContainerDied","Data":"1c2dcfd2d2b359ed2e869200e415689acd5fb5191723f7e332b7aa18e3ad74ad"}
Sep 16 16:47:05.508990 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:05.508971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc5664dbc-8dnf5" event={"ID":"24eeec43-4bf5-4edd-816c-4f2b351eb077","Type":"ContainerStarted","Data":"6e2b725415cc02e1292490a6517c9da62f1f969ea0a228b7d230000291db3c5d"}
Sep 16 16:47:05.509076 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:05.509012    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cc5664dbc-8dnf5" event={"ID":"24eeec43-4bf5-4edd-816c-4f2b351eb077","Type":"ContainerStarted","Data":"00bf1f56130e7f92b9270ac5fb060fc519470a5b642e55c6e850e8daca5a9db3"}
Sep 16 16:47:05.542256 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:05.542206    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cc5664dbc-8dnf5" podStartSLOduration=2.5421847680000003 podStartE2EDuration="2.542184768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:47:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:47:05.541649616 +0000 UTC m=+641.038522741" watchObservedRunningTime="2026-09-16 16:47:05.542184768 +0000 UTC m=+641.039057892"
Sep 16 16:47:06.515472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:06.515435    2577 generic.go:356] "Generic (PLEG): container finished" podID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerID="94020f056bd14e92073ec2e2b4ccf19a17f262d74297a7740069e88956692c65" exitCode=0
Sep 16 16:47:06.515911 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:06.515541    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79" event={"ID":"f2d0e62f-b0f5-4896-af1e-14358df91148","Type":"ContainerDied","Data":"94020f056bd14e92073ec2e2b4ccf19a17f262d74297a7740069e88956692c65"}
Sep 16 16:47:06.517387 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:06.517366    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerID="a70a3c65047d9858f61702d0e11f5e0cf14c29c606a189f66668e7cac12fde64" exitCode=0
Sep 16 16:47:06.517489 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:06.517458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk" event={"ID":"ae6a3124-b093-41c7-be5c-ae04bfbbb556","Type":"ContainerDied","Data":"a70a3c65047d9858f61702d0e11f5e0cf14c29c606a189f66668e7cac12fde64"}
Sep 16 16:47:06.519439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:06.519418    2577 generic.go:356] "Generic (PLEG): container finished" podID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerID="5bcc89eb33024db52901fe6b26ee333d0813a07766744269ad6997f32555ee3a" exitCode=0
Sep 16 16:47:06.519550 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:06.519504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw" event={"ID":"a918f3e8-6895-4353-9639-50fc7dc066cc","Type":"ContainerDied","Data":"5bcc89eb33024db52901fe6b26ee333d0813a07766744269ad6997f32555ee3a"}
Sep 16 16:47:06.521839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:06.521820    2577 generic.go:356] "Generic (PLEG): container finished" podID="a4660274-0926-422a-859e-d8d906c5bf07" containerID="07f4e91b3d35fb8a613042ed2bb0c190ca06d7eadc1ed4ed7449729ef3f58bf1" exitCode=0
Sep 16 16:47:06.521948 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:06.521885    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8" event={"ID":"a4660274-0926-422a-859e-d8d906c5bf07","Type":"ContainerDied","Data":"07f4e91b3d35fb8a613042ed2bb0c190ca06d7eadc1ed4ed7449729ef3f58bf1"}
Sep 16 16:47:07.528619 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.528584    2577 generic.go:356] "Generic (PLEG): container finished" podID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerID="f9b7b0311230ae037310c92fb999921a236cfec05a2c08abf4c04266a5833b54" exitCode=0
Sep 16 16:47:07.529047 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.528664    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw" event={"ID":"a918f3e8-6895-4353-9639-50fc7dc066cc","Type":"ContainerDied","Data":"f9b7b0311230ae037310c92fb999921a236cfec05a2c08abf4c04266a5833b54"}
Sep 16 16:47:07.530705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.530685    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerID="38d30a778fd02c25fe65859b90d043fa54b6c8af0b4d29f5089f645c8acf0e00" exitCode=0
Sep 16 16:47:07.530801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.530766    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk" event={"ID":"ae6a3124-b093-41c7-be5c-ae04bfbbb556","Type":"ContainerDied","Data":"38d30a778fd02c25fe65859b90d043fa54b6c8af0b4d29f5089f645c8acf0e00"}
Sep 16 16:47:07.678638 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.678614    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:07.681524 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.681507    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:07.798307 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.798220    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwz9j\" (UniqueName: \"kubernetes.io/projected/f2d0e62f-b0f5-4896-af1e-14358df91148-kube-api-access-gwz9j\") pod \"f2d0e62f-b0f5-4896-af1e-14358df91148\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") "
Sep 16 16:47:07.798307 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.798273    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrvv5\" (UniqueName: \"kubernetes.io/projected/a4660274-0926-422a-859e-d8d906c5bf07-kube-api-access-rrvv5\") pod \"a4660274-0926-422a-859e-d8d906c5bf07\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") "
Sep 16 16:47:07.798551 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.798314    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-bundle\") pod \"f2d0e62f-b0f5-4896-af1e-14358df91148\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") "
Sep 16 16:47:07.798551 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.798351    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-util\") pod \"a4660274-0926-422a-859e-d8d906c5bf07\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") "
Sep 16 16:47:07.798551 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.798391    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-util\") pod \"f2d0e62f-b0f5-4896-af1e-14358df91148\" (UID: \"f2d0e62f-b0f5-4896-af1e-14358df91148\") "
Sep 16 16:47:07.798551 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.798438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-bundle\") pod \"a4660274-0926-422a-859e-d8d906c5bf07\" (UID: \"a4660274-0926-422a-859e-d8d906c5bf07\") "
Sep 16 16:47:07.799105 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.799066    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-bundle" (OuterVolumeSpecName: "bundle") pod "a4660274-0926-422a-859e-d8d906c5bf07" (UID: "a4660274-0926-422a-859e-d8d906c5bf07"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:47:07.799234 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.799125    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-bundle" (OuterVolumeSpecName: "bundle") pod "f2d0e62f-b0f5-4896-af1e-14358df91148" (UID: "f2d0e62f-b0f5-4896-af1e-14358df91148"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:47:07.800451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.800425    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4660274-0926-422a-859e-d8d906c5bf07-kube-api-access-rrvv5" (OuterVolumeSpecName: "kube-api-access-rrvv5") pod "a4660274-0926-422a-859e-d8d906c5bf07" (UID: "a4660274-0926-422a-859e-d8d906c5bf07"). InnerVolumeSpecName "kube-api-access-rrvv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:47:07.800757 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.800730    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d0e62f-b0f5-4896-af1e-14358df91148-kube-api-access-gwz9j" (OuterVolumeSpecName: "kube-api-access-gwz9j") pod "f2d0e62f-b0f5-4896-af1e-14358df91148" (UID: "f2d0e62f-b0f5-4896-af1e-14358df91148"). InnerVolumeSpecName "kube-api-access-gwz9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:47:07.804206 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.804175    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-util" (OuterVolumeSpecName: "util") pod "a4660274-0926-422a-859e-d8d906c5bf07" (UID: "a4660274-0926-422a-859e-d8d906c5bf07"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:47:07.805785 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.805762    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-util" (OuterVolumeSpecName: "util") pod "f2d0e62f-b0f5-4896-af1e-14358df91148" (UID: "f2d0e62f-b0f5-4896-af1e-14358df91148"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:47:07.899487 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.899457    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwz9j\" (UniqueName: \"kubernetes.io/projected/f2d0e62f-b0f5-4896-af1e-14358df91148-kube-api-access-gwz9j\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:07.899487 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.899481    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrvv5\" (UniqueName: \"kubernetes.io/projected/a4660274-0926-422a-859e-d8d906c5bf07-kube-api-access-rrvv5\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:07.899652 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.899491    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:07.899652 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.899503    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-util\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:07.899652 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.899510    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2d0e62f-b0f5-4896-af1e-14358df91148-util\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:07.899652 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:07.899517    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4660274-0926-422a-859e-d8d906c5bf07-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:08.537975 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.537945    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79"
Sep 16 16:47:08.538387 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.537971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847awnl79" event={"ID":"f2d0e62f-b0f5-4896-af1e-14358df91148","Type":"ContainerDied","Data":"0516353de9e8e6126ad10adda1c828c94ead4aea0f6303784c68c2e0b1c6cb49"}
Sep 16 16:47:08.538387 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.538023    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0516353de9e8e6126ad10adda1c828c94ead4aea0f6303784c68c2e0b1c6cb49"
Sep 16 16:47:08.539978 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.539956    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8"
Sep 16 16:47:08.540087 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.539976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9hzh8" event={"ID":"a4660274-0926-422a-859e-d8d906c5bf07","Type":"ContainerDied","Data":"b5f065c95c4c73c3068286abb223a4bf71cc1db4ad4aea4a601da547541c09fa"}
Sep 16 16:47:08.540087 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.540016    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5f065c95c4c73c3068286abb223a4bf71cc1db4ad4aea4a601da547541c09fa"
Sep 16 16:47:08.670047 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.670012    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:08.701425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.701398    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:08.807927 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.807830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-bundle\") pod \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") "
Sep 16 16:47:08.807927 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.807896    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt79m\" (UniqueName: \"kubernetes.io/projected/a918f3e8-6895-4353-9639-50fc7dc066cc-kube-api-access-zt79m\") pod \"a918f3e8-6895-4353-9639-50fc7dc066cc\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") "
Sep 16 16:47:08.807927 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.807921    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpw45\" (UniqueName: \"kubernetes.io/projected/ae6a3124-b093-41c7-be5c-ae04bfbbb556-kube-api-access-cpw45\") pod \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") "
Sep 16 16:47:08.808183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.807941    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-bundle\") pod \"a918f3e8-6895-4353-9639-50fc7dc066cc\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") "
Sep 16 16:47:08.808183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.808168    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-util\") pod \"a918f3e8-6895-4353-9639-50fc7dc066cc\" (UID: \"a918f3e8-6895-4353-9639-50fc7dc066cc\") "
Sep 16 16:47:08.808300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.808210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-util\") pod \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\" (UID: \"ae6a3124-b093-41c7-be5c-ae04bfbbb556\") "
Sep 16 16:47:08.808384 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.808345    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-bundle" (OuterVolumeSpecName: "bundle") pod "ae6a3124-b093-41c7-be5c-ae04bfbbb556" (UID: "ae6a3124-b093-41c7-be5c-ae04bfbbb556"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:47:08.808384 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.808368    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-bundle" (OuterVolumeSpecName: "bundle") pod "a918f3e8-6895-4353-9639-50fc7dc066cc" (UID: "a918f3e8-6895-4353-9639-50fc7dc066cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:47:08.808567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.808520    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:08.808644 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.808569    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:08.810108 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.810079    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a918f3e8-6895-4353-9639-50fc7dc066cc-kube-api-access-zt79m" (OuterVolumeSpecName: "kube-api-access-zt79m") pod "a918f3e8-6895-4353-9639-50fc7dc066cc" (UID: "a918f3e8-6895-4353-9639-50fc7dc066cc"). InnerVolumeSpecName "kube-api-access-zt79m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:47:08.810248 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.810225    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae6a3124-b093-41c7-be5c-ae04bfbbb556-kube-api-access-cpw45" (OuterVolumeSpecName: "kube-api-access-cpw45") pod "ae6a3124-b093-41c7-be5c-ae04bfbbb556" (UID: "ae6a3124-b093-41c7-be5c-ae04bfbbb556"). InnerVolumeSpecName "kube-api-access-cpw45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:47:08.814218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.814198    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-util" (OuterVolumeSpecName: "util") pod "a918f3e8-6895-4353-9639-50fc7dc066cc" (UID: "a918f3e8-6895-4353-9639-50fc7dc066cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:47:08.814454 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.814435    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-util" (OuterVolumeSpecName: "util") pod "ae6a3124-b093-41c7-be5c-ae04bfbbb556" (UID: "ae6a3124-b093-41c7-be5c-ae04bfbbb556"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 16:47:08.908983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.908952    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt79m\" (UniqueName: \"kubernetes.io/projected/a918f3e8-6895-4353-9639-50fc7dc066cc-kube-api-access-zt79m\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:08.908983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.908977    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpw45\" (UniqueName: \"kubernetes.io/projected/ae6a3124-b093-41c7-be5c-ae04bfbbb556-kube-api-access-cpw45\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:08.908983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.908988    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a918f3e8-6895-4353-9639-50fc7dc066cc-util\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:08.909183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:08.908998    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae6a3124-b093-41c7-be5c-ae04bfbbb556-util\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:09.546373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:09.546336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk" event={"ID":"ae6a3124-b093-41c7-be5c-ae04bfbbb556","Type":"ContainerDied","Data":"a14ff02b50ba227a76a416785c19f69e0540fe49b6d1cd98d60ecc5978ed4627"}
Sep 16 16:47:09.546735 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:09.546386    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a14ff02b50ba227a76a416785c19f69e0540fe49b6d1cd98d60ecc5978ed4627"
Sep 16 16:47:09.546735 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:09.546442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d79j9wk"
Sep 16 16:47:09.548603 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:09.548575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw" event={"ID":"a918f3e8-6895-4353-9639-50fc7dc066cc","Type":"ContainerDied","Data":"6f829274fedc07ed1f301c031dbe5bacf1f69bf0e041a1fb5a20bb531e6837c2"}
Sep 16 16:47:09.548710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:09.548611    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6w4pnw"
Sep 16 16:47:09.548710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:09.548619    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f829274fedc07ed1f301c031dbe5bacf1f69bf0e041a1fb5a20bb531e6837c2"
Sep 16 16:47:13.505858 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:13.505825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:14.235279 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:14.235242    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:14.235440 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:14.235316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:14.239939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:14.239908    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:14.577574 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:14.577476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cc5664dbc-8dnf5"
Sep 16 16:47:14.632992 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:14.632964    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-576c4478fc-vqs76"]
Sep 16 16:47:14.794096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:14.794063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:14.965584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:14.965552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:15.066802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:15.066757    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:17.621129 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:17.621087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:19.478041 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:19.478003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:19.741092 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:19.741013    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:19.852371 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:19.852322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:20.419255 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:20.419188    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:21.159705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:21.159671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:21.864889 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:21.864855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:22.000768 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:22.000729    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:23.615972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:23.615922    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:24.045867 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:24.045833    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:25.861607 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:25.861576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:25.963926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:25.963881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:27.400406 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:27.400366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:27.500628 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:27.500597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:29.429889 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:29.429853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:29.531440 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:29.531410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:31.293035 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.292999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:31.294140 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294094    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"]
Sep 16 16:47:31.294437 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294422    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerName="util"
Sep 16 16:47:31.294482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294438    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerName="util"
Sep 16 16:47:31.294482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294450    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerName="util"
Sep 16 16:47:31.294482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294456    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerName="util"
Sep 16 16:47:31.294482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294463    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4660274-0926-422a-859e-d8d906c5bf07" containerName="extract"
Sep 16 16:47:31.294482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294469    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4660274-0926-422a-859e-d8d906c5bf07" containerName="extract"
Sep 16 16:47:31.294482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294481    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerName="util"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294486    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerName="util"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294493    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294498    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294509    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4660274-0926-422a-859e-d8d906c5bf07" containerName="util"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294514    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4660274-0926-422a-859e-d8d906c5bf07" containerName="util"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294519    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerName="pull"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294536    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerName="pull"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294541    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4660274-0926-422a-859e-d8d906c5bf07" containerName="pull"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294547    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4660274-0926-422a-859e-d8d906c5bf07" containerName="pull"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294554    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerName="pull"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294559    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerName="pull"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294565    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294570    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294579    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294583    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294592    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerName="pull"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294596    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerName="pull"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294665    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a918f3e8-6895-4353-9639-50fc7dc066cc" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294677    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4660274-0926-422a-859e-d8d906c5bf07" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294684    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae6a3124-b093-41c7-be5c-ae04bfbbb556" containerName="extract"
Sep 16 16:47:31.294695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.294691    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2d0e62f-b0f5-4896-af1e-14358df91148" containerName="extract"
Sep 16 16:47:31.296826 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.296809    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.299205 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.299186    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 16 16:47:31.299330 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.299209    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 16:47:31.299330 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.299258    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kkrlh\""
Sep 16 16:47:31.299330 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.299210    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 16:47:31.299451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.299422    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 16 16:47:31.310544 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.307709    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"]
Sep 16 16:47:31.398039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.398008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0e098df-1e6b-4ab3-9f83-70247205fda0-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.398190 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.398052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0e098df-1e6b-4ab3-9f83-70247205fda0-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.398190 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.398072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzmsw\" (UniqueName: \"kubernetes.io/projected/d0e098df-1e6b-4ab3-9f83-70247205fda0-kube-api-access-lzmsw\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.476570 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.476508    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:31.498634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.498606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0e098df-1e6b-4ab3-9f83-70247205fda0-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.498730 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.498662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0e098df-1e6b-4ab3-9f83-70247205fda0-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.498730 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.498686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzmsw\" (UniqueName: \"kubernetes.io/projected/d0e098df-1e6b-4ab3-9f83-70247205fda0-kube-api-access-lzmsw\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.499163 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.499132    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d0e098df-1e6b-4ab3-9f83-70247205fda0-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.500971 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.500951    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d0e098df-1e6b-4ab3-9f83-70247205fda0-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.507171 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.507147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzmsw\" (UniqueName: \"kubernetes.io/projected/d0e098df-1e6b-4ab3-9f83-70247205fda0-kube-api-access-lzmsw\") pod \"kuadrant-console-plugin-fbdd4f445-4jdk8\" (UID: \"d0e098df-1e6b-4ab3-9f83-70247205fda0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.606669 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.606608    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"
Sep 16 16:47:31.876138 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:31.876109    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8"]
Sep 16 16:47:31.877601 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:47:31.877576    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0e098df_1e6b_4ab3_9f83_70247205fda0.slice/crio-53c5038808b0ff2d139f1fa5bf18da30485921ddf9a57869e2ad6264e0a0a42f WatchSource:0}: Error finding container 53c5038808b0ff2d139f1fa5bf18da30485921ddf9a57869e2ad6264e0a0a42f: Status 404 returned error can't find the container with id 53c5038808b0ff2d139f1fa5bf18da30485921ddf9a57869e2ad6264e0a0a42f
Sep 16 16:47:32.669574 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:32.669537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8" event={"ID":"d0e098df-1e6b-4ab3-9f83-70247205fda0","Type":"ContainerStarted","Data":"53c5038808b0ff2d139f1fa5bf18da30485921ddf9a57869e2ad6264e0a0a42f"}
Sep 16 16:47:35.225634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:35.225585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:35.324060 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:35.324010    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:35.423365 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:35.423316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:37.702755 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:37.702714    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8" event={"ID":"d0e098df-1e6b-4ab3-9f83-70247205fda0","Type":"ContainerStarted","Data":"2dc54ae9af01d18bbb1c944bd1fea2f31a76adecbb61da3b4115f3bade0bf179"}
Sep 16 16:47:37.726196 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:37.726144    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-4jdk8" podStartSLOduration=1.92645624 podStartE2EDuration="6.726123708s" totalImagesPullingTime="4.799667468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:47:37.725338885 +0000 UTC m=+673.222212020" watchObservedRunningTime="2026-09-16 16:47:37.726123708 +0000 UTC m=+673.222996831"
Sep 16 16:47:39.654288 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.654225    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-576c4478fc-vqs76" podUID="d8efa0b5-6558-4814-8522-ccf0adc50529" containerName="console" containerID="cri-o://c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05" gracePeriod=15
Sep 16 16:47:39.903038 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.903009    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:47:39.966462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966395    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-serving-cert\") pod \"d8efa0b5-6558-4814-8522-ccf0adc50529\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") "
Sep 16 16:47:39.966462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966426    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-trusted-ca-bundle\") pod \"d8efa0b5-6558-4814-8522-ccf0adc50529\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") "
Sep 16 16:47:39.966462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966445    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-service-ca\") pod \"d8efa0b5-6558-4814-8522-ccf0adc50529\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") "
Sep 16 16:47:39.966462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966463    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-console-config\") pod \"d8efa0b5-6558-4814-8522-ccf0adc50529\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") "
Sep 16 16:47:39.966797 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966504    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtm56\" (UniqueName: \"kubernetes.io/projected/d8efa0b5-6558-4814-8522-ccf0adc50529-kube-api-access-vtm56\") pod \"d8efa0b5-6558-4814-8522-ccf0adc50529\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") "
Sep 16 16:47:39.966797 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-oauth-config\") pod \"d8efa0b5-6558-4814-8522-ccf0adc50529\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") "
Sep 16 16:47:39.966797 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966597    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-oauth-serving-cert\") pod \"d8efa0b5-6558-4814-8522-ccf0adc50529\" (UID: \"d8efa0b5-6558-4814-8522-ccf0adc50529\") "
Sep 16 16:47:39.966973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966833    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-service-ca" (OuterVolumeSpecName: "service-ca") pod "d8efa0b5-6558-4814-8522-ccf0adc50529" (UID: "d8efa0b5-6558-4814-8522-ccf0adc50529"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:47:39.966973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-console-config" (OuterVolumeSpecName: "console-config") pod "d8efa0b5-6558-4814-8522-ccf0adc50529" (UID: "d8efa0b5-6558-4814-8522-ccf0adc50529"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:47:39.966973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966896    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:39.966973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.966883    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d8efa0b5-6558-4814-8522-ccf0adc50529" (UID: "d8efa0b5-6558-4814-8522-ccf0adc50529"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:47:39.967176 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.967131    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d8efa0b5-6558-4814-8522-ccf0adc50529" (UID: "d8efa0b5-6558-4814-8522-ccf0adc50529"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:47:39.968518 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.968496    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d8efa0b5-6558-4814-8522-ccf0adc50529" (UID: "d8efa0b5-6558-4814-8522-ccf0adc50529"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:47:39.968667 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.968644    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d8efa0b5-6558-4814-8522-ccf0adc50529" (UID: "d8efa0b5-6558-4814-8522-ccf0adc50529"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:47:39.968715 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:39.968665    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8efa0b5-6558-4814-8522-ccf0adc50529-kube-api-access-vtm56" (OuterVolumeSpecName: "kube-api-access-vtm56") pod "d8efa0b5-6558-4814-8522-ccf0adc50529" (UID: "d8efa0b5-6558-4814-8522-ccf0adc50529"). InnerVolumeSpecName "kube-api-access-vtm56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:47:40.067882 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.067843    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-serving-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:40.067882 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.067878    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-trusted-ca-bundle\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:40.067882 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.067889    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-console-config\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:40.068089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.067898    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtm56\" (UniqueName: \"kubernetes.io/projected/d8efa0b5-6558-4814-8522-ccf0adc50529-kube-api-access-vtm56\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:40.068089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.067907    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8efa0b5-6558-4814-8522-ccf0adc50529-console-oauth-config\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:40.068089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.067916    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8efa0b5-6558-4814-8522-ccf0adc50529-oauth-serving-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:47:40.718935 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.718903    2577 generic.go:356] "Generic (PLEG): container finished" podID="d8efa0b5-6558-4814-8522-ccf0adc50529" containerID="c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05" exitCode=0
Sep 16 16:47:40.719359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.718949    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-576c4478fc-vqs76"
Sep 16 16:47:40.719359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.718993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c4478fc-vqs76" event={"ID":"d8efa0b5-6558-4814-8522-ccf0adc50529","Type":"ContainerDied","Data":"c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05"}
Sep 16 16:47:40.719359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.719045    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-576c4478fc-vqs76" event={"ID":"d8efa0b5-6558-4814-8522-ccf0adc50529","Type":"ContainerDied","Data":"c55528ca7795068363ee4897a3c43b5c6a24e88f412015ac02e3576d871a25cf"}
Sep 16 16:47:40.719359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.719064    2577 scope.go:117] "RemoveContainer" containerID="c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05"
Sep 16 16:47:40.728802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.728784    2577 scope.go:117] "RemoveContainer" containerID="c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05"
Sep 16 16:47:40.729069 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:47:40.729052    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05\": container with ID starting with c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05 not found: ID does not exist" containerID="c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05"
Sep 16 16:47:40.729119 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.729076    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05"} err="failed to get container status \"c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05\": rpc error: code = NotFound desc = could not find container \"c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05\": container with ID starting with c1fa3e90463acc4bba389fcf6f47181f2722a49ceba4f2b1ba9d78c013bfda05 not found: ID does not exist"
Sep 16 16:47:40.761758 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.761729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-576c4478fc-vqs76"]
Sep 16 16:47:40.776017 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.775983    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-576c4478fc-vqs76"]
Sep 16 16:47:40.977463 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:40.977377    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8efa0b5-6558-4814-8522-ccf0adc50529" path="/var/lib/kubelet/pods/d8efa0b5-6558-4814-8522-ccf0adc50529/volumes"
Sep 16 16:47:44.063867 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:44.063823    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:44.162229 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:44.162184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:46.668658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:46.668622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:46.859428 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:46.859395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:48.626590 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:48.626559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:49.456570 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:49.456507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:49.555158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:49.555119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:54.242544 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:54.242494    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:54.344245 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:54.344206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:54.547098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:54.547013    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:54.689363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:54.689327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:55.493210 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:55.493177    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:47:55.585691 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:47:55.585659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:03.054047 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:03.054011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:03.154093 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:03.154046    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:10.698036 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:10.697996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:11.130642 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:11.130536    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:27.131005 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.130968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:27.229750 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.229714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:27.332615 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.332581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:27.920631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.920598    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pdclv"]
Sep 16 16:48:27.920937 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.920924    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8efa0b5-6558-4814-8522-ccf0adc50529" containerName="console"
Sep 16 16:48:27.920987 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.920938    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8efa0b5-6558-4814-8522-ccf0adc50529" containerName="console"
Sep 16 16:48:27.921027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.920986    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8efa0b5-6558-4814-8522-ccf0adc50529" containerName="console"
Sep 16 16:48:27.924884 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.924858    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv"
Sep 16 16:48:27.927909 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.927890    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-96js7\""
Sep 16 16:48:27.932760 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.932738    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pdclv"]
Sep 16 16:48:27.955886 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.955843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:27.979498 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:27.979475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd9dc\" (UniqueName: \"kubernetes.io/projected/fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b-kube-api-access-jd9dc\") pod \"authorino-7bcfd8f9cd-pdclv\" (UID: \"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv"
Sep 16 16:48:28.044973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.044932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:28.049211 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.049182    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-642gj"]
Sep 16 16:48:28.052359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.052344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-642gj"
Sep 16 16:48:28.058014 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.057994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-642gj"]
Sep 16 16:48:28.080492 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.080471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vkc5\" (UniqueName: \"kubernetes.io/projected/394d3d14-b5f7-4131-b04b-95ff42e12b71-kube-api-access-9vkc5\") pod \"authorino-59c974bd48-642gj\" (UID: \"394d3d14-b5f7-4131-b04b-95ff42e12b71\") " pod="kuadrant-system/authorino-59c974bd48-642gj"
Sep 16 16:48:28.080608 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.080507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd9dc\" (UniqueName: \"kubernetes.io/projected/fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b-kube-api-access-jd9dc\") pod \"authorino-7bcfd8f9cd-pdclv\" (UID: \"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv"
Sep 16 16:48:28.087988 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.087970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd9dc\" (UniqueName: \"kubernetes.io/projected/fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b-kube-api-access-jd9dc\") pod \"authorino-7bcfd8f9cd-pdclv\" (UID: \"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv"
Sep 16 16:48:28.181434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.181344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vkc5\" (UniqueName: \"kubernetes.io/projected/394d3d14-b5f7-4131-b04b-95ff42e12b71-kube-api-access-9vkc5\") pod \"authorino-59c974bd48-642gj\" (UID: \"394d3d14-b5f7-4131-b04b-95ff42e12b71\") " pod="kuadrant-system/authorino-59c974bd48-642gj"
Sep 16 16:48:28.189357 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.189336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vkc5\" (UniqueName: \"kubernetes.io/projected/394d3d14-b5f7-4131-b04b-95ff42e12b71-kube-api-access-9vkc5\") pod \"authorino-59c974bd48-642gj\" (UID: \"394d3d14-b5f7-4131-b04b-95ff42e12b71\") " pod="kuadrant-system/authorino-59c974bd48-642gj"
Sep 16 16:48:28.235133 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.235109    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv"
Sep 16 16:48:28.361794 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.361755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-642gj"
Sep 16 16:48:28.503061 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.502875    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pdclv"]
Sep 16 16:48:28.505574 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:48:28.505524    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa55a7cf_55ea_4a1e_bdbc_71bf6e98006b.slice/crio-313d03feb2bbe33de743a48061ac0db47f2968c9d72198262f7473430ce48f75 WatchSource:0}: Error finding container 313d03feb2bbe33de743a48061ac0db47f2968c9d72198262f7473430ce48f75: Status 404 returned error can't find the container with id 313d03feb2bbe33de743a48061ac0db47f2968c9d72198262f7473430ce48f75
Sep 16 16:48:28.630540 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.630502    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-642gj"]
Sep 16 16:48:28.632377 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:48:28.632352    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod394d3d14_b5f7_4131_b04b_95ff42e12b71.slice/crio-7ebeb3b347d80041a0f1da5552171ad93a9ad73078129fc360896715f3406336 WatchSource:0}: Error finding container 7ebeb3b347d80041a0f1da5552171ad93a9ad73078129fc360896715f3406336: Status 404 returned error can't find the container with id 7ebeb3b347d80041a0f1da5552171ad93a9ad73078129fc360896715f3406336
Sep 16 16:48:28.957439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.957402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv" event={"ID":"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b","Type":"ContainerStarted","Data":"313d03feb2bbe33de743a48061ac0db47f2968c9d72198262f7473430ce48f75"}
Sep 16 16:48:28.958539 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:28.958502    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-642gj" event={"ID":"394d3d14-b5f7-4131-b04b-95ff42e12b71","Type":"ContainerStarted","Data":"7ebeb3b347d80041a0f1da5552171ad93a9ad73078129fc360896715f3406336"}
Sep 16 16:48:31.980309 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:31.980268    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv" event={"ID":"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b","Type":"ContainerStarted","Data":"8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b"}
Sep 16 16:48:31.981900 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:31.981873    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-642gj" event={"ID":"394d3d14-b5f7-4131-b04b-95ff42e12b71","Type":"ContainerStarted","Data":"abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71"}
Sep 16 16:48:31.997259 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:31.997213    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv" podStartSLOduration=2.085637902 podStartE2EDuration="4.997201287s" totalImagesPullingTime="2.911563385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:48:31.995219126 +0000 UTC m=+727.492092252" watchObservedRunningTime="2026-09-16 16:48:31.997201287 +0000 UTC m=+727.494074414"
Sep 16 16:48:32.033077 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:32.033017    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-642gj" podStartSLOduration=1.237143629 podStartE2EDuration="4.033000293s" totalImagesPullingTime="2.795856664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:48:32.010069846 +0000 UTC m=+727.506942967" watchObservedRunningTime="2026-09-16 16:48:32.033000293 +0000 UTC m=+727.529873417"
Sep 16 16:48:32.035623 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:32.035600    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pdclv"]
Sep 16 16:48:33.992054 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:33.992012    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv" podUID="fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b" containerName="authorino" containerID="cri-o://8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b" gracePeriod=30
Sep 16 16:48:34.229484 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.229459    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv"
Sep 16 16:48:34.333364 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.333276    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd9dc\" (UniqueName: \"kubernetes.io/projected/fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b-kube-api-access-jd9dc\") pod \"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b\" (UID: \"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b\") "
Sep 16 16:48:34.335476 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.335449    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b-kube-api-access-jd9dc" (OuterVolumeSpecName: "kube-api-access-jd9dc") pod "fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b" (UID: "fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b"). InnerVolumeSpecName "kube-api-access-jd9dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:48:34.434939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.434890    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd9dc\" (UniqueName: \"kubernetes.io/projected/fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b-kube-api-access-jd9dc\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:48:34.997573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.997501    2577 generic.go:356] "Generic (PLEG): container finished" podID="fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b" containerID="8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b" exitCode=0
Sep 16 16:48:34.997573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.997563    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv"
Sep 16 16:48:34.997573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.997568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv" event={"ID":"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b","Type":"ContainerDied","Data":"8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b"}
Sep 16 16:48:34.998048 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.997605    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-pdclv" event={"ID":"fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b","Type":"ContainerDied","Data":"313d03feb2bbe33de743a48061ac0db47f2968c9d72198262f7473430ce48f75"}
Sep 16 16:48:34.998048 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:34.997626    2577 scope.go:117] "RemoveContainer" containerID="8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b"
Sep 16 16:48:35.006274 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:35.006255    2577 scope.go:117] "RemoveContainer" containerID="8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b"
Sep 16 16:48:35.006568 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:48:35.006548    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b\": container with ID starting with 8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b not found: ID does not exist" containerID="8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b"
Sep 16 16:48:35.006624 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:35.006578    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b"} err="failed to get container status \"8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b\": rpc error: code = NotFound desc = could not find container \"8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b\": container with ID starting with 8d8254411f189dbe13286a4d1919c2eb9b6ff823cb1d070e97a53157d8cb1c1b not found: ID does not exist"
Sep 16 16:48:35.014500 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:35.014480    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pdclv"]
Sep 16 16:48:35.018824 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:35.018806    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-pdclv"]
Sep 16 16:48:36.977888 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:36.977855    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b" path="/var/lib/kubelet/pods/fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b/volumes"
Sep 16 16:48:44.443487 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:44.443444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:44.542792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:44.542760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:45.146045 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:45.146004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:56.902026 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:56.901988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:57.117554 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.117511    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-t444n"]
Sep 16 16:48:57.117896 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.117883    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b" containerName="authorino"
Sep 16 16:48:57.117945 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.117898    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b" containerName="authorino"
Sep 16 16:48:57.117980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.117955    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa55a7cf-55ea-4a1e-bdbc-71bf6e98006b" containerName="authorino"
Sep 16 16:48:57.120751 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.120737    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-t444n"
Sep 16 16:48:57.128922 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.128886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:48:57.129974 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.129953    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-t444n"]
Sep 16 16:48:57.182771 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.182693    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-t444n"]
Sep 16 16:48:57.182970 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:48:57.182950    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2hkmq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-t444n" podUID="3eb733cb-ba3b-4029-be4b-834a0f0f3162"
Sep 16 16:48:57.224357 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.224324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hkmq\" (UniqueName: \"kubernetes.io/projected/3eb733cb-ba3b-4029-be4b-834a0f0f3162-kube-api-access-2hkmq\") pod \"authorino-557f876f69-t444n\" (UID: \"3eb733cb-ba3b-4029-be4b-834a0f0f3162\") " pod="kuadrant-system/authorino-557f876f69-t444n"
Sep 16 16:48:57.325741 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.325705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hkmq\" (UniqueName: \"kubernetes.io/projected/3eb733cb-ba3b-4029-be4b-834a0f0f3162-kube-api-access-2hkmq\") pod \"authorino-557f876f69-t444n\" (UID: \"3eb733cb-ba3b-4029-be4b-834a0f0f3162\") " pod="kuadrant-system/authorino-557f876f69-t444n"
Sep 16 16:48:57.336423 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.336400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hkmq\" (UniqueName: \"kubernetes.io/projected/3eb733cb-ba3b-4029-be4b-834a0f0f3162-kube-api-access-2hkmq\") pod \"authorino-557f876f69-t444n\" (UID: \"3eb733cb-ba3b-4029-be4b-834a0f0f3162\") " pod="kuadrant-system/authorino-557f876f69-t444n"
Sep 16 16:48:57.376311 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.376282    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7746576469-2x4kk"]
Sep 16 16:48:57.379563 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.379546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:48:57.381920 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.381903    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 16:48:57.388005 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.387979    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7746576469-2x4kk"]
Sep 16 16:48:57.527840 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.527755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce2e6a25-afe9-4b03-b414-ad107b338f60-tls-cert\") pod \"authorino-7746576469-2x4kk\" (UID: \"ce2e6a25-afe9-4b03-b414-ad107b338f60\") " pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:48:57.527840 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.527810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzw4h\" (UniqueName: \"kubernetes.io/projected/ce2e6a25-afe9-4b03-b414-ad107b338f60-kube-api-access-pzw4h\") pod \"authorino-7746576469-2x4kk\" (UID: \"ce2e6a25-afe9-4b03-b414-ad107b338f60\") " pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:48:57.628705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.628669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzw4h\" (UniqueName: \"kubernetes.io/projected/ce2e6a25-afe9-4b03-b414-ad107b338f60-kube-api-access-pzw4h\") pod \"authorino-7746576469-2x4kk\" (UID: \"ce2e6a25-afe9-4b03-b414-ad107b338f60\") " pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:48:57.628873 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.628749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce2e6a25-afe9-4b03-b414-ad107b338f60-tls-cert\") pod \"authorino-7746576469-2x4kk\" (UID: \"ce2e6a25-afe9-4b03-b414-ad107b338f60\") " pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:48:57.631138 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.631113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce2e6a25-afe9-4b03-b414-ad107b338f60-tls-cert\") pod \"authorino-7746576469-2x4kk\" (UID: \"ce2e6a25-afe9-4b03-b414-ad107b338f60\") " pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:48:57.636258 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.636232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzw4h\" (UniqueName: \"kubernetes.io/projected/ce2e6a25-afe9-4b03-b414-ad107b338f60-kube-api-access-pzw4h\") pod \"authorino-7746576469-2x4kk\" (UID: \"ce2e6a25-afe9-4b03-b414-ad107b338f60\") " pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:48:57.688953 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.688927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:48:57.961766 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:57.961737    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7746576469-2x4kk"]
Sep 16 16:48:57.963322 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:48:57.963294    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce2e6a25_afe9_4b03_b414_ad107b338f60.slice/crio-f236c9ea0aeab8d7c0290cd326d761d6904834848f4801c84b95d8a05f18e3bc WatchSource:0}: Error finding container f236c9ea0aeab8d7c0290cd326d761d6904834848f4801c84b95d8a05f18e3bc: Status 404 returned error can't find the container with id f236c9ea0aeab8d7c0290cd326d761d6904834848f4801c84b95d8a05f18e3bc
Sep 16 16:48:58.113147 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:58.113114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7746576469-2x4kk" event={"ID":"ce2e6a25-afe9-4b03-b414-ad107b338f60","Type":"ContainerStarted","Data":"f236c9ea0aeab8d7c0290cd326d761d6904834848f4801c84b95d8a05f18e3bc"}
Sep 16 16:48:58.113315 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:58.113187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-t444n"
Sep 16 16:48:58.121204 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:58.121181    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-t444n"
Sep 16 16:48:58.233638 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:58.233545    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hkmq\" (UniqueName: \"kubernetes.io/projected/3eb733cb-ba3b-4029-be4b-834a0f0f3162-kube-api-access-2hkmq\") pod \"3eb733cb-ba3b-4029-be4b-834a0f0f3162\" (UID: \"3eb733cb-ba3b-4029-be4b-834a0f0f3162\") "
Sep 16 16:48:58.235559 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:58.235515    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb733cb-ba3b-4029-be4b-834a0f0f3162-kube-api-access-2hkmq" (OuterVolumeSpecName: "kube-api-access-2hkmq") pod "3eb733cb-ba3b-4029-be4b-834a0f0f3162" (UID: "3eb733cb-ba3b-4029-be4b-834a0f0f3162"). InnerVolumeSpecName "kube-api-access-2hkmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:48:58.334668 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:58.334638    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hkmq\" (UniqueName: \"kubernetes.io/projected/3eb733cb-ba3b-4029-be4b-834a0f0f3162-kube-api-access-2hkmq\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:48:59.121169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.121070    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7746576469-2x4kk" event={"ID":"ce2e6a25-afe9-4b03-b414-ad107b338f60","Type":"ContainerStarted","Data":"75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e"}
Sep 16 16:48:59.121169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.121091    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-t444n"
Sep 16 16:48:59.142128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.142082    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7746576469-2x4kk" podStartSLOduration=1.249841848 podStartE2EDuration="2.14206673s" totalImagesPullingTime="892.224882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:48:59.141037327 +0000 UTC m=+754.637910464" watchObservedRunningTime="2026-09-16 16:48:59.14206673 +0000 UTC m=+754.638939881"
Sep 16 16:48:59.167567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.167524    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-t444n"]
Sep 16 16:48:59.172879 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.172827    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-642gj"]
Sep 16 16:48:59.173145 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.173064    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-642gj" podUID="394d3d14-b5f7-4131-b04b-95ff42e12b71" containerName="authorino" containerID="cri-o://abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71" gracePeriod=30
Sep 16 16:48:59.175721 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.175695    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-t444n"]
Sep 16 16:48:59.418693 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.418669    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-642gj"
Sep 16 16:48:59.547801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.547760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vkc5\" (UniqueName: \"kubernetes.io/projected/394d3d14-b5f7-4131-b04b-95ff42e12b71-kube-api-access-9vkc5\") pod \"394d3d14-b5f7-4131-b04b-95ff42e12b71\" (UID: \"394d3d14-b5f7-4131-b04b-95ff42e12b71\") "
Sep 16 16:48:59.549815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.549792    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/394d3d14-b5f7-4131-b04b-95ff42e12b71-kube-api-access-9vkc5" (OuterVolumeSpecName: "kube-api-access-9vkc5") pod "394d3d14-b5f7-4131-b04b-95ff42e12b71" (UID: "394d3d14-b5f7-4131-b04b-95ff42e12b71"). InnerVolumeSpecName "kube-api-access-9vkc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:48:59.648501 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:48:59.648468    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vkc5\" (UniqueName: \"kubernetes.io/projected/394d3d14-b5f7-4131-b04b-95ff42e12b71-kube-api-access-9vkc5\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:49:00.128116 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.128081    2577 generic.go:356] "Generic (PLEG): container finished" podID="394d3d14-b5f7-4131-b04b-95ff42e12b71" containerID="abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71" exitCode=0
Sep 16 16:49:00.128569 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.128126    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-642gj"
Sep 16 16:49:00.128569 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.128166    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-642gj" event={"ID":"394d3d14-b5f7-4131-b04b-95ff42e12b71","Type":"ContainerDied","Data":"abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71"}
Sep 16 16:49:00.128569 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.128209    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-642gj" event={"ID":"394d3d14-b5f7-4131-b04b-95ff42e12b71","Type":"ContainerDied","Data":"7ebeb3b347d80041a0f1da5552171ad93a9ad73078129fc360896715f3406336"}
Sep 16 16:49:00.128569 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.128227    2577 scope.go:117] "RemoveContainer" containerID="abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71"
Sep 16 16:49:00.137087 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.137071    2577 scope.go:117] "RemoveContainer" containerID="abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71"
Sep 16 16:49:00.137372 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:49:00.137354    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71\": container with ID starting with abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71 not found: ID does not exist" containerID="abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71"
Sep 16 16:49:00.137427 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.137381    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71"} err="failed to get container status \"abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71\": rpc error: code = NotFound desc = could not find container \"abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71\": container with ID starting with abbfa528da033f0b97cce68d475bbffc17d87e2b8bd5d91049bf892e0b0c3f71 not found: ID does not exist"
Sep 16 16:49:00.148573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.148550    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-642gj"]
Sep 16 16:49:00.152582 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.152559    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-642gj"]
Sep 16 16:49:00.977794 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.977762    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="394d3d14-b5f7-4131-b04b-95ff42e12b71" path="/var/lib/kubelet/pods/394d3d14-b5f7-4131-b04b-95ff42e12b71/volumes"
Sep 16 16:49:00.978069 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:00.978055    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eb733cb-ba3b-4029-be4b-834a0f0f3162" path="/var/lib/kubelet/pods/3eb733cb-ba3b-4029-be4b-834a0f0f3162/volumes"
Sep 16 16:49:03.748164 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.748124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:03.845221 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.845174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:03.945197 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.945152    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:03.946675 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.946653    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-r6jxd"]
Sep 16 16:49:03.947000 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.946986    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="394d3d14-b5f7-4131-b04b-95ff42e12b71" containerName="authorino"
Sep 16 16:49:03.947047 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.947001    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="394d3d14-b5f7-4131-b04b-95ff42e12b71" containerName="authorino"
Sep 16 16:49:03.947089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.947061    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="394d3d14-b5f7-4131-b04b-95ff42e12b71" containerName="authorino"
Sep 16 16:49:03.951582 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.951563    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:03.955477 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.955457    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8kj7n\""
Sep 16 16:49:03.955628 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.955459    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 16:49:03.955771 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.955755    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 16:49:03.955872 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.955838    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 16:49:03.964642 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:03.964619    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-r6jxd"]
Sep 16 16:49:04.018338 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.018247    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:04.096303 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.096268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-metrics-certs\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.096484 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.096386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glmx6\" (UniqueName: \"kubernetes.io/projected/007df39e-e454-47cb-8658-364e1ef34b60-kube-api-access-glmx6\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.096484 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.096431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-webhook-cert\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.197222 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.197181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glmx6\" (UniqueName: \"kubernetes.io/projected/007df39e-e454-47cb-8658-364e1ef34b60-kube-api-access-glmx6\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.197222 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.197226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-webhook-cert\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.197470 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.197376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-metrics-certs\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.199725 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.199696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-metrics-certs\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.199832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.199696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-webhook-cert\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.205564 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.205523    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glmx6\" (UniqueName: \"kubernetes.io/projected/007df39e-e454-47cb-8658-364e1ef34b60-kube-api-access-glmx6\") pod \"maas-controller-6bb99c5f94-r6jxd\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") " pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.261275 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.261242    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:04.533998 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:04.533971    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-r6jxd"]
Sep 16 16:49:04.536066 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:49:04.536040    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007df39e_e454_47cb_8658_364e1ef34b60.slice/crio-d7ba777d3624d3ed0e4efb30d4ccc06f5b6abef8b669d1ae243a87a6c6ecd267 WatchSource:0}: Error finding container d7ba777d3624d3ed0e4efb30d4ccc06f5b6abef8b669d1ae243a87a6c6ecd267: Status 404 returned error can't find the container with id d7ba777d3624d3ed0e4efb30d4ccc06f5b6abef8b669d1ae243a87a6c6ecd267
Sep 16 16:49:05.157957 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:05.157919    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd" event={"ID":"007df39e-e454-47cb-8658-364e1ef34b60","Type":"ContainerStarted","Data":"d7ba777d3624d3ed0e4efb30d4ccc06f5b6abef8b669d1ae243a87a6c6ecd267"}
Sep 16 16:49:07.514678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:07.514641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:07.616731 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:07.616642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:08.179188 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:08.179153    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd" event={"ID":"007df39e-e454-47cb-8658-364e1ef34b60","Type":"ContainerStarted","Data":"b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79"}
Sep 16 16:49:08.179411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:08.179246    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:08.198034 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:08.197970    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd" podStartSLOduration=2.438917511 podStartE2EDuration="5.197955341s" totalImagesPullingTime="2.75903783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:49:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:49:08.196199281 +0000 UTC m=+763.693072407" watchObservedRunningTime="2026-09-16 16:49:08.197955341 +0000 UTC m=+763.694828464"
Sep 16 16:49:19.186809 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:19.186775    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 16:49:38.963003 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:38.962948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:39.148243 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:39.148204    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:40.378766 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:40.378713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:40.479917 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:40.479884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:40.899862 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:40.899827    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:41.002816 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.002778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:41.036326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.036298    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"]
Sep 16 16:49:41.043573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.043554    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:41.046252 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.046229    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 16:49:41.046355 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.046253    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-g4st5\""
Sep 16 16:49:41.054253 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.054225    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"]
Sep 16 16:49:41.113598 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.113571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:41.220458 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.220378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d1d3ce7-f283-4b11-bf5e-14ae89090257-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-7tvs6\" (UID: \"0d1d3ce7-f283-4b11-bf5e-14ae89090257\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:41.220458 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.220413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvnmn\" (UniqueName: \"kubernetes.io/projected/0d1d3ce7-f283-4b11-bf5e-14ae89090257-kube-api-access-bvnmn\") pod \"payload-pre-processing-59fbc47bf8-7tvs6\" (UID: \"0d1d3ce7-f283-4b11-bf5e-14ae89090257\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:41.321218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.321183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d1d3ce7-f283-4b11-bf5e-14ae89090257-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-7tvs6\" (UID: \"0d1d3ce7-f283-4b11-bf5e-14ae89090257\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:41.321218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.321219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvnmn\" (UniqueName: \"kubernetes.io/projected/0d1d3ce7-f283-4b11-bf5e-14ae89090257-kube-api-access-bvnmn\") pod \"payload-pre-processing-59fbc47bf8-7tvs6\" (UID: \"0d1d3ce7-f283-4b11-bf5e-14ae89090257\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:41.321914 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.321891    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d1d3ce7-f283-4b11-bf5e-14ae89090257-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-7tvs6\" (UID: \"0d1d3ce7-f283-4b11-bf5e-14ae89090257\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:41.333923 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.333891    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvnmn\" (UniqueName: \"kubernetes.io/projected/0d1d3ce7-f283-4b11-bf5e-14ae89090257-kube-api-access-bvnmn\") pod \"payload-pre-processing-59fbc47bf8-7tvs6\" (UID: \"0d1d3ce7-f283-4b11-bf5e-14ae89090257\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:41.355587 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.355547    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:41.667897 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:41.667870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"]
Sep 16 16:49:41.669113 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:49:41.669086    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1d3ce7_f283_4b11_bf5e_14ae89090257.slice/crio-73815554604c63cd1f47408f3de267defdaa0fd69c57ee7b27c8f006f46a7159 WatchSource:0}: Error finding container 73815554604c63cd1f47408f3de267defdaa0fd69c57ee7b27c8f006f46a7159: Status 404 returned error can't find the container with id 73815554604c63cd1f47408f3de267defdaa0fd69c57ee7b27c8f006f46a7159
Sep 16 16:49:42.354323 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:42.354281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6" event={"ID":"0d1d3ce7-f283-4b11-bf5e-14ae89090257","Type":"ContainerStarted","Data":"73815554604c63cd1f47408f3de267defdaa0fd69c57ee7b27c8f006f46a7159"}
Sep 16 16:49:43.361424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:43.361326    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6" event={"ID":"0d1d3ce7-f283-4b11-bf5e-14ae89090257","Type":"ContainerStarted","Data":"75d99af74d970a6f211b69fa011c0da980deb1ff9dcf990e707913c506900aa2"}
Sep 16 16:49:43.361424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:43.361394    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:43.381920 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:43.381873    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6" podStartSLOduration=1.038881426 podStartE2EDuration="2.381857313s" totalImagesPullingTime="1.342975887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:49:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:49:43.378971906 +0000 UTC m=+798.875845039" watchObservedRunningTime="2026-09-16 16:49:43.381857313 +0000 UTC m=+798.878730437"
Sep 16 16:49:50.917579 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:50.917435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:51.020958 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:51.020921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:51.590545 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:51.590490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:51.654699 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:51.654662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:54.368569 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:54.368511    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-7tvs6"
Sep 16 16:49:55.706021 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.705987    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"]
Sep 16 16:49:55.709448 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.709431    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.711939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.711915    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 16:49:55.712693 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.712676    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 16:49:55.712767 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.712676    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 16 16:49:55.712767 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.712676    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cxtbk\""
Sep 16 16:49:55.719694 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.719673    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"]
Sep 16 16:49:55.742400 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.742370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:49:55.751570 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.751543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.751664 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.751603    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.751728 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.751710    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv6sc\" (UniqueName: \"kubernetes.io/projected/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-kube-api-access-rv6sc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.751769 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.751756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.751807 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.751793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.751850 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.751836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.852929 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.852892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.853081 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.852966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv6sc\" (UniqueName: \"kubernetes.io/projected/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-kube-api-access-rv6sc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.853081 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.853000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.853081 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.853031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.853081 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.853057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.853290 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.853092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.853428 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.853403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.853501 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.853467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.853588 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.853517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.855286 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.855259    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.855491 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.855472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:55.860317 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:55.860295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv6sc\" (UniqueName: \"kubernetes.io/projected/2b1c20fb-34b5-4ce7-bbb2-bf13659390aa-kube-api-access-rv6sc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7\" (UID: \"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:56.021869 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:56.021777    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:49:56.294955 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:56.294404    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"]
Sep 16 16:49:56.296638 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:49:56.296604    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1c20fb_34b5_4ce7_bbb2_bf13659390aa.slice/crio-3c5d6bf7db0e3d0a10f2999a492a0f62ec779af9fdd00a0913784b151493237b WatchSource:0}: Error finding container 3c5d6bf7db0e3d0a10f2999a492a0f62ec779af9fdd00a0913784b151493237b: Status 404 returned error can't find the container with id 3c5d6bf7db0e3d0a10f2999a492a0f62ec779af9fdd00a0913784b151493237b
Sep 16 16:49:56.428646 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:56.428606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7" event={"ID":"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa","Type":"ContainerStarted","Data":"3c5d6bf7db0e3d0a10f2999a492a0f62ec779af9fdd00a0913784b151493237b"}
Sep 16 16:49:57.437257 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:57.437229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7" event={"ID":"2b1c20fb-34b5-4ce7-bbb2-bf13659390aa","Type":"ContainerStarted","Data":"e9dfd1d6709163f8c2b232de8695b7299826b7de83d2f82b5fad74c2c019cc07"}
Sep 16 16:49:57.454643 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:49:57.454595    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7" podStartSLOduration=1.4110537220000001 podStartE2EDuration="2.454581537s" totalImagesPullingTime="1.043527815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:49:57.4533479 +0000 UTC m=+812.950221025" watchObservedRunningTime="2026-09-16 16:49:57.454581537 +0000 UTC m=+812.951454660"
Sep 16 16:50:06.022927 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:06.022882    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:50:06.023356 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:06.022971    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:50:06.035473 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:06.035448    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:50:06.496511 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:06.496484    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7"
Sep 16 16:50:09.606701 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:09.606667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:50:26.196834 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:26.196787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:50:31.481044 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.481001    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"]
Sep 16 16:50:31.485876 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.485853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.488247 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.488223    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 16 16:50:31.494984 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.494953    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"]
Sep 16 16:50:31.517415 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.517379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:50:31.672486 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.672453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.672486 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.672493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b75976-64e4-4b90-960d-a9c94a85245a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.672740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.672515    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.672740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.672656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.672740 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.672734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.672888 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.672771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llvjc\" (UniqueName: \"kubernetes.io/projected/16b75976-64e4-4b90-960d-a9c94a85245a-kube-api-access-llvjc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llvjc\" (UniqueName: \"kubernetes.io/projected/16b75976-64e4-4b90-960d-a9c94a85245a-kube-api-access-llvjc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773699 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b75976-64e4-4b90-960d-a9c94a85245a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773699 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773699 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773853 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773920 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.773920 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.773904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.775721 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.775703    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16b75976-64e4-4b90-960d-a9c94a85245a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.775863 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.775845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16b75976-64e4-4b90-960d-a9c94a85245a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.781899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.781880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llvjc\" (UniqueName: \"kubernetes.io/projected/16b75976-64e4-4b90-960d-a9c94a85245a-kube-api-access-llvjc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-w5n79\" (UID: \"16b75976-64e4-4b90-960d-a9c94a85245a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:31.798365 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:31.798349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:32.073063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:32.073037    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"]
Sep 16 16:50:32.074820 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:50:32.074795    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16b75976_64e4_4b90_960d_a9c94a85245a.slice/crio-a136b3fd064feb488533a0c539ed8291360dceb6a7718acadc250d8d299523f7 WatchSource:0}: Error finding container a136b3fd064feb488533a0c539ed8291360dceb6a7718acadc250d8d299523f7: Status 404 returned error can't find the container with id a136b3fd064feb488533a0c539ed8291360dceb6a7718acadc250d8d299523f7
Sep 16 16:50:32.622545 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:32.622430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79" event={"ID":"16b75976-64e4-4b90-960d-a9c94a85245a","Type":"ContainerStarted","Data":"64a5d0b966d9f42bdbf12b16e00ee86129a7763af74d886cf12790f4359688fb"}
Sep 16 16:50:32.622545 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:32.622484    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79" event={"ID":"16b75976-64e4-4b90-960d-a9c94a85245a","Type":"ContainerStarted","Data":"a136b3fd064feb488533a0c539ed8291360dceb6a7718acadc250d8d299523f7"}
Sep 16 16:50:32.642475 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:32.642421    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79" podStartSLOduration=1.439608735 podStartE2EDuration="1.642404241s" totalImagesPullingTime="202.795506ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:50:32.641066454 +0000 UTC m=+848.137939592" watchObservedRunningTime="2026-09-16 16:50:32.642404241 +0000 UTC m=+848.139277365"
Sep 16 16:50:41.799466 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:41.799427    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:41.799918 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:41.799516    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:41.812095 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:41.812071    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:42.188473 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.188439    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"]
Sep 16 16:50:42.192129 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.192106    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.194568 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.194518    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 16 16:50:42.203137 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.203114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"]
Sep 16 16:50:42.219871 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.219839    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:50:42.252273 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.252245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.252420 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.252279    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.252420 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.252298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f2bda8-6656-4e2e-a890-823e9dba257e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.252420 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.252407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v8fk\" (UniqueName: \"kubernetes.io/projected/62f2bda8-6656-4e2e-a890-823e9dba257e-kube-api-access-4v8fk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.252560 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.252483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.252622 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.252605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.352976 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.352946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.353118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.353010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.353118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.353056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.353118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.353085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.353118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.353111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f2bda8-6656-4e2e-a890-823e9dba257e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.353305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.353152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v8fk\" (UniqueName: \"kubernetes.io/projected/62f2bda8-6656-4e2e-a890-823e9dba257e-kube-api-access-4v8fk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.353428 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.353408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.353428 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.353417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.353567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.353506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.355456 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.355433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62f2bda8-6656-4e2e-a890-823e9dba257e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.355717 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.355697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62f2bda8-6656-4e2e-a890-823e9dba257e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.361504 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.361484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v8fk\" (UniqueName: \"kubernetes.io/projected/62f2bda8-6656-4e2e-a890-823e9dba257e-kube-api-access-4v8fk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg\" (UID: \"62f2bda8-6656-4e2e-a890-823e9dba257e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.503522 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.503435    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:42.687437 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.687407    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-w5n79"
Sep 16 16:50:42.778155 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:42.778126    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"]
Sep 16 16:50:42.779512 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:50:42.779486    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f2bda8_6656_4e2e_a890_823e9dba257e.slice/crio-44e7251848115adba0be4ba985d9543ce3dda442bcd353980f19f6e90ee9f8ed WatchSource:0}: Error finding container 44e7251848115adba0be4ba985d9543ce3dda442bcd353980f19f6e90ee9f8ed: Status 404 returned error can't find the container with id 44e7251848115adba0be4ba985d9543ce3dda442bcd353980f19f6e90ee9f8ed
Sep 16 16:50:43.701428 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:43.701391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg" event={"ID":"62f2bda8-6656-4e2e-a890-823e9dba257e","Type":"ContainerStarted","Data":"498654a98e3a1d5f4731635c735eb234d07d5148a11bfddf24bea2a2dcb0d6c1"}
Sep 16 16:50:43.701904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:43.701447    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg" event={"ID":"62f2bda8-6656-4e2e-a890-823e9dba257e","Type":"ContainerStarted","Data":"44e7251848115adba0be4ba985d9543ce3dda442bcd353980f19f6e90ee9f8ed"}
Sep 16 16:50:43.722083 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:43.722036    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg" podStartSLOduration=1.526659974 podStartE2EDuration="1.72201943s" totalImagesPullingTime="195.359456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:50:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:50:43.719612768 +0000 UTC m=+859.216485891" watchObservedRunningTime="2026-09-16 16:50:43.72201943 +0000 UTC m=+859.218892585"
Sep 16 16:50:46.020341 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:46.020299    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:50:52.504226 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:52.504180    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:52.504623 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:52.504248    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:52.516566 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:52.516516    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:52.760978 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:52.760895    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg"
Sep 16 16:50:57.119761 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:50:57.119724    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:51:24.916279 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:24.916170    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:51:24.924055 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:24.918521    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:51:36.011514 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:36.011476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:51:44.207160 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.207123    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6fdd987d44-5vc68"]
Sep 16 16:51:44.210483 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.210464    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fdd987d44-5vc68"
Sep 16 16:51:44.221637 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.221613    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fdd987d44-5vc68"]
Sep 16 16:51:44.297782 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.297747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z5bk\" (UniqueName: \"kubernetes.io/projected/4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4-kube-api-access-7z5bk\") pod \"authorino-6fdd987d44-5vc68\" (UID: \"4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4\") " pod="kuadrant-system/authorino-6fdd987d44-5vc68"
Sep 16 16:51:44.297945 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.297824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4-tls-cert\") pod \"authorino-6fdd987d44-5vc68\" (UID: \"4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4\") " pod="kuadrant-system/authorino-6fdd987d44-5vc68"
Sep 16 16:51:44.398752 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.398700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4-tls-cert\") pod \"authorino-6fdd987d44-5vc68\" (UID: \"4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4\") " pod="kuadrant-system/authorino-6fdd987d44-5vc68"
Sep 16 16:51:44.398926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.398790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z5bk\" (UniqueName: \"kubernetes.io/projected/4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4-kube-api-access-7z5bk\") pod \"authorino-6fdd987d44-5vc68\" (UID: \"4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4\") " pod="kuadrant-system/authorino-6fdd987d44-5vc68"
Sep 16 16:51:44.401071 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.401046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4-tls-cert\") pod \"authorino-6fdd987d44-5vc68\" (UID: \"4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4\") " pod="kuadrant-system/authorino-6fdd987d44-5vc68"
Sep 16 16:51:44.407887 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.407858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z5bk\" (UniqueName: \"kubernetes.io/projected/4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4-kube-api-access-7z5bk\") pod \"authorino-6fdd987d44-5vc68\" (UID: \"4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4\") " pod="kuadrant-system/authorino-6fdd987d44-5vc68"
Sep 16 16:51:44.520661 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.520599    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fdd987d44-5vc68"
Sep 16 16:51:44.792403 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:44.792367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fdd987d44-5vc68"]
Sep 16 16:51:44.794360 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:51:44.794333    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4eed6d7d_7c03_4389_a6f4_9e14a1b32ac4.slice/crio-cd9f725c249875e5099c30fa920968f0427ce1c33e53c17ddbb8a42a63b3d48a WatchSource:0}: Error finding container cd9f725c249875e5099c30fa920968f0427ce1c33e53c17ddbb8a42a63b3d48a: Status 404 returned error can't find the container with id cd9f725c249875e5099c30fa920968f0427ce1c33e53c17ddbb8a42a63b3d48a
Sep 16 16:51:45.027160 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:45.027128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fdd987d44-5vc68" event={"ID":"4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4","Type":"ContainerStarted","Data":"cd9f725c249875e5099c30fa920968f0427ce1c33e53c17ddbb8a42a63b3d48a"}
Sep 16 16:51:46.036782 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.036729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fdd987d44-5vc68" event={"ID":"4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4","Type":"ContainerStarted","Data":"dc8e2bbb390f364fccf9bd611bd06b4cfcd98e3b7b2f9506df88a5bbad6aa8cb"}
Sep 16 16:51:46.057393 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.057336    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6fdd987d44-5vc68" podStartSLOduration=1.246569431 podStartE2EDuration="2.057316655s" totalImagesPullingTime="810.747224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:51:46.055795869 +0000 UTC m=+921.552668994" watchObservedRunningTime="2026-09-16 16:51:46.057316655 +0000 UTC m=+921.554189778"
Sep 16 16:51:46.097424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.097390    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7746576469-2x4kk"]
Sep 16 16:51:46.097679 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.097645    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7746576469-2x4kk" podUID="ce2e6a25-afe9-4b03-b414-ad107b338f60" containerName="authorino" containerID="cri-o://75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e" gracePeriod=30
Sep 16 16:51:46.354610 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.354568    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:51:46.417343 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.417313    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce2e6a25-afe9-4b03-b414-ad107b338f60-tls-cert\") pod \"ce2e6a25-afe9-4b03-b414-ad107b338f60\" (UID: \"ce2e6a25-afe9-4b03-b414-ad107b338f60\") "
Sep 16 16:51:46.417513 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.417442    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzw4h\" (UniqueName: \"kubernetes.io/projected/ce2e6a25-afe9-4b03-b414-ad107b338f60-kube-api-access-pzw4h\") pod \"ce2e6a25-afe9-4b03-b414-ad107b338f60\" (UID: \"ce2e6a25-afe9-4b03-b414-ad107b338f60\") "
Sep 16 16:51:46.419986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.419955    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2e6a25-afe9-4b03-b414-ad107b338f60-kube-api-access-pzw4h" (OuterVolumeSpecName: "kube-api-access-pzw4h") pod "ce2e6a25-afe9-4b03-b414-ad107b338f60" (UID: "ce2e6a25-afe9-4b03-b414-ad107b338f60"). InnerVolumeSpecName "kube-api-access-pzw4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:51:46.427986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.427950    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce2e6a25-afe9-4b03-b414-ad107b338f60-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ce2e6a25-afe9-4b03-b414-ad107b338f60" (UID: "ce2e6a25-afe9-4b03-b414-ad107b338f60"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 16:51:46.518495 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.518459    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzw4h\" (UniqueName: \"kubernetes.io/projected/ce2e6a25-afe9-4b03-b414-ad107b338f60-kube-api-access-pzw4h\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:51:46.518495 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.518489    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ce2e6a25-afe9-4b03-b414-ad107b338f60-tls-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:51:46.641397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.641346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:51:46.735541 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:46.735497    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:51:47.043851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.043812    2577 generic.go:356] "Generic (PLEG): container finished" podID="ce2e6a25-afe9-4b03-b414-ad107b338f60" containerID="75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e" exitCode=0
Sep 16 16:51:47.044342 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.043869    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7746576469-2x4kk"
Sep 16 16:51:47.044342 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.043914    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7746576469-2x4kk" event={"ID":"ce2e6a25-afe9-4b03-b414-ad107b338f60","Type":"ContainerDied","Data":"75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e"}
Sep 16 16:51:47.044342 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.043967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7746576469-2x4kk" event={"ID":"ce2e6a25-afe9-4b03-b414-ad107b338f60","Type":"ContainerDied","Data":"f236c9ea0aeab8d7c0290cd326d761d6904834848f4801c84b95d8a05f18e3bc"}
Sep 16 16:51:47.044342 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.043989    2577 scope.go:117] "RemoveContainer" containerID="75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e"
Sep 16 16:51:47.054021 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.054001    2577 scope.go:117] "RemoveContainer" containerID="75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e"
Sep 16 16:51:47.054293 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:51:47.054273    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e\": container with ID starting with 75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e not found: ID does not exist" containerID="75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e"
Sep 16 16:51:47.054343 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.054301    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e"} err="failed to get container status \"75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e\": rpc error: code = NotFound desc = could not find container \"75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e\": container with ID starting with 75c75402128c6d72b7f61a87e65bd0c9750bf86f7d36cf44e3ac692ac6229a7e not found: ID does not exist"
Sep 16 16:51:47.064063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.064035    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7746576469-2x4kk"]
Sep 16 16:51:47.072046 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.072016    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7746576469-2x4kk"]
Sep 16 16:51:47.685885 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.685849    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:51:47.784706 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:47.784671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:51:48.983807 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:51:48.983765    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce2e6a25-afe9-4b03-b414-ad107b338f60" path="/var/lib/kubelet/pods/ce2e6a25-afe9-4b03-b414-ad107b338f60/volumes"
Sep 16 16:52:03.172523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:03.172467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:03.287762 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:03.287699    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:03.395166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:03.395128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:03.652295 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:03.652256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:03.851175 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:03.851138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:03.947906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:03.947801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:04.051632 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:04.051598    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:04.465352 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:04.465314    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:04.603523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:04.603487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:04.702601 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:04.702523    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:05.020315 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.020281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:05.075769 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.075735    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7929-98e07f"]
Sep 16 16:52:05.076134 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.076117    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2e6a25-afe9-4b03-b414-ad107b338f60" containerName="authorino"
Sep 16 16:52:05.076218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.076136    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2e6a25-afe9-4b03-b414-ad107b338f60" containerName="authorino"
Sep 16 16:52:05.076281 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.076273    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce2e6a25-afe9-4b03-b414-ad107b338f60" containerName="authorino"
Sep 16 16:52:05.079362 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.079342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7929-98e07f"
Sep 16 16:52:05.081986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.081961    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rcrsh\""
Sep 16 16:52:05.086360 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.086337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7929-98e07f"]
Sep 16 16:52:05.183783 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.183750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hks78\" (UniqueName: \"kubernetes.io/projected/0fe6ad70-1027-476c-9144-b80dd3a25224-kube-api-access-hks78\") pod \"test-curl-7929-98e07f\" (UID: \"0fe6ad70-1027-476c-9144-b80dd3a25224\") " pod="opendatahub/test-curl-7929-98e07f"
Sep 16 16:52:05.285376 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.285292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hks78\" (UniqueName: \"kubernetes.io/projected/0fe6ad70-1027-476c-9144-b80dd3a25224-kube-api-access-hks78\") pod \"test-curl-7929-98e07f\" (UID: \"0fe6ad70-1027-476c-9144-b80dd3a25224\") " pod="opendatahub/test-curl-7929-98e07f"
Sep 16 16:52:05.315361 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.315329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hks78\" (UniqueName: \"kubernetes.io/projected/0fe6ad70-1027-476c-9144-b80dd3a25224-kube-api-access-hks78\") pod \"test-curl-7929-98e07f\" (UID: \"0fe6ad70-1027-476c-9144-b80dd3a25224\") " pod="opendatahub/test-curl-7929-98e07f"
Sep 16 16:52:05.390292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.390252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7929-98e07f"
Sep 16 16:52:05.670133 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.670103    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7929-98e07f"]
Sep 16 16:52:05.673093 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:05.673056    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fe6ad70_1027_476c_9144_b80dd3a25224.slice/crio-76a7009e195c0d60fc0436cd5e570029ba920c7a98e693be097e7b322922e19e WatchSource:0}: Error finding container 76a7009e195c0d60fc0436cd5e570029ba920c7a98e693be097e7b322922e19e: Status 404 returned error can't find the container with id 76a7009e195c0d60fc0436cd5e570029ba920c7a98e693be097e7b322922e19e
Sep 16 16:52:05.675267 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:05.675248    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 16:52:06.150269 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:06.150234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7929-98e07f" event={"ID":"0fe6ad70-1027-476c-9144-b80dd3a25224","Type":"ContainerStarted","Data":"76a7009e195c0d60fc0436cd5e570029ba920c7a98e693be097e7b322922e19e"}
Sep 16 16:52:06.254175 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:06.254139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:06.348148 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:06.348107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:07.260823 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:07.260721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:07.348385 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:07.348350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:07.617674 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:07.617587    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:07.691144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:07.691109    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:07.802801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:07.802759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:08.141107 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.141071    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:08.163464 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.163438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7929-98e07f" event={"ID":"0fe6ad70-1027-476c-9144-b80dd3a25224","Type":"ContainerStarted","Data":"9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe"}
Sep 16 16:52:08.203642 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.203593    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7929-98e07f" podStartSLOduration=0.855260217 podStartE2EDuration="3.203577758s" totalImagesPullingTime="2.348317541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:08.201229348 +0000 UTC m=+943.698102475" watchObservedRunningTime="2026-09-16 16:52:08.203577758 +0000 UTC m=+943.700450882"
Sep 16 16:52:08.293678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.293610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:08.339233 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.339207    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"]
Sep 16 16:52:08.342604 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.342587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:08.348401 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.348384    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-eb2ab2-dockercfg-lh5cp\""
Sep 16 16:52:08.348471 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.348382    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-eb2ab2\""
Sep 16 16:52:08.363831 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.363806    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"]
Sep 16 16:52:08.415635 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.415610    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3875cd77-6d4c-4809-b377-17f7bae294ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf\" (UID: \"3875cd77-6d4c-4809-b377-17f7bae294ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:08.415758 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.415657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bj9q\" (UniqueName: \"kubernetes.io/projected/3875cd77-6d4c-4809-b377-17f7bae294ca-kube-api-access-8bj9q\") pod \"payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf\" (UID: \"3875cd77-6d4c-4809-b377-17f7bae294ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:08.422171 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.422147    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"]
Sep 16 16:52:08.425523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.425507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:08.443790 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.443765    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"]
Sep 16 16:52:08.516999 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.516965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrs6f\" (UniqueName: \"kubernetes.io/projected/f02a3453-e414-4683-b3d9-245070c0c33c-kube-api-access-hrs6f\") pod \"payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9\" (UID: \"f02a3453-e414-4683-b3d9-245070c0c33c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:08.517181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.517024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f02a3453-e414-4683-b3d9-245070c0c33c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9\" (UID: \"f02a3453-e414-4683-b3d9-245070c0c33c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:08.517181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.517142    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3875cd77-6d4c-4809-b377-17f7bae294ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf\" (UID: \"3875cd77-6d4c-4809-b377-17f7bae294ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:08.517181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.517172    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bj9q\" (UniqueName: \"kubernetes.io/projected/3875cd77-6d4c-4809-b377-17f7bae294ca-kube-api-access-8bj9q\") pod \"payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf\" (UID: \"3875cd77-6d4c-4809-b377-17f7bae294ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:08.517938 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.517913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3875cd77-6d4c-4809-b377-17f7bae294ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf\" (UID: \"3875cd77-6d4c-4809-b377-17f7bae294ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:08.528094 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.528073    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bj9q\" (UniqueName: \"kubernetes.io/projected/3875cd77-6d4c-4809-b377-17f7bae294ca-kube-api-access-8bj9q\") pod \"payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf\" (UID: \"3875cd77-6d4c-4809-b377-17f7bae294ca\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:08.617744 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.617668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f02a3453-e414-4683-b3d9-245070c0c33c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9\" (UID: \"f02a3453-e414-4683-b3d9-245070c0c33c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:08.617931 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.617823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrs6f\" (UniqueName: \"kubernetes.io/projected/f02a3453-e414-4683-b3d9-245070c0c33c-kube-api-access-hrs6f\") pod \"payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9\" (UID: \"f02a3453-e414-4683-b3d9-245070c0c33c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:08.618347 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.618326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f02a3453-e414-4683-b3d9-245070c0c33c-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9\" (UID: \"f02a3453-e414-4683-b3d9-245070c0c33c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:08.625341 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.625318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrs6f\" (UniqueName: \"kubernetes.io/projected/f02a3453-e414-4683-b3d9-245070c0c33c-kube-api-access-hrs6f\") pod \"payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9\" (UID: \"f02a3453-e414-4683-b3d9-245070c0c33c\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:08.652451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.652425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:08.735682 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.735598    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:08.858928 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.858901    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7929-98e07f"]
Sep 16 16:52:08.863369 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.863340    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7929-98e07f"]
Sep 16 16:52:08.927934 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.927904    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"]
Sep 16 16:52:08.929140 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:08.929112    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3875cd77_6d4c_4809_b377_17f7bae294ca.slice/crio-80bcff005cc68b41498a66a2e304baf66b6f8735b526add5137143e4908a6750 WatchSource:0}: Error finding container 80bcff005cc68b41498a66a2e304baf66b6f8735b526add5137143e4908a6750: Status 404 returned error can't find the container with id 80bcff005cc68b41498a66a2e304baf66b6f8735b526add5137143e4908a6750
Sep 16 16:52:08.980638 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.980617    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7929-cb03b5"]
Sep 16 16:52:08.981089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.981068    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fe6ad70-1027-476c-9144-b80dd3a25224" containerName="test-curl-7929-98e07f"
Sep 16 16:52:08.981170 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.981092    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fe6ad70-1027-476c-9144-b80dd3a25224" containerName="test-curl-7929-98e07f"
Sep 16 16:52:08.981214 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.981172    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fe6ad70-1027-476c-9144-b80dd3a25224" containerName="test-curl-7929-98e07f"
Sep 16 16:52:08.984640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.984619    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7929-cb03b5"
Sep 16 16:52:08.993849 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:08.993818    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7929-cb03b5"]
Sep 16 16:52:09.011939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.011902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:09.028747 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.028720    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"]
Sep 16 16:52:09.030452 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:09.030429    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02a3453_e414_4683_b3d9_245070c0c33c.slice/crio-68b0305fed1d8a4995a77b9ac0e3aa7c71037f6cd70570d6de87277811ecf31a WatchSource:0}: Error finding container 68b0305fed1d8a4995a77b9ac0e3aa7c71037f6cd70570d6de87277811ecf31a: Status 404 returned error can't find the container with id 68b0305fed1d8a4995a77b9ac0e3aa7c71037f6cd70570d6de87277811ecf31a
Sep 16 16:52:09.099936 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.099911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:09.122807 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.122778    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxvzb\" (UniqueName: \"kubernetes.io/projected/34504b49-745e-4091-b543-01aed933d991-kube-api-access-gxvzb\") pod \"test-curl-7929-cb03b5\" (UID: \"34504b49-745e-4091-b543-01aed933d991\") " pod="opendatahub/test-curl-7929-cb03b5"
Sep 16 16:52:09.170855 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.170817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf" event={"ID":"3875cd77-6d4c-4809-b377-17f7bae294ca","Type":"ContainerStarted","Data":"9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d"}
Sep 16 16:52:09.170997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.170879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf" event={"ID":"3875cd77-6d4c-4809-b377-17f7bae294ca","Type":"ContainerStarted","Data":"80bcff005cc68b41498a66a2e304baf66b6f8735b526add5137143e4908a6750"}
Sep 16 16:52:09.171071 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.171002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:09.172515 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.172490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9" event={"ID":"f02a3453-e414-4683-b3d9-245070c0c33c","Type":"ContainerStarted","Data":"2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd"}
Sep 16 16:52:09.172640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.172567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9" event={"ID":"f02a3453-e414-4683-b3d9-245070c0c33c","Type":"ContainerStarted","Data":"68b0305fed1d8a4995a77b9ac0e3aa7c71037f6cd70570d6de87277811ecf31a"}
Sep 16 16:52:09.172640 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.172598    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:09.172640 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:52:09.172613    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7929-98e07f" container="test-curl-7929-98e07f"
Sep 16 16:52:09.187365 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.187291    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf" podStartSLOduration=1.187277037 podStartE2EDuration="1.187277037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:09.186321832 +0000 UTC m=+944.683194958" watchObservedRunningTime="2026-09-16 16:52:09.187277037 +0000 UTC m=+944.684150190"
Sep 16 16:52:09.204506 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.204465    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9" podStartSLOduration=1.2044524779999999 podStartE2EDuration="1.204452478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:09.203262763 +0000 UTC m=+944.700135890" watchObservedRunningTime="2026-09-16 16:52:09.204452478 +0000 UTC m=+944.701325602"
Sep 16 16:52:09.223803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.223772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxvzb\" (UniqueName: \"kubernetes.io/projected/34504b49-745e-4091-b543-01aed933d991-kube-api-access-gxvzb\") pod \"test-curl-7929-cb03b5\" (UID: \"34504b49-745e-4091-b543-01aed933d991\") " pod="opendatahub/test-curl-7929-cb03b5"
Sep 16 16:52:09.231542 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.231509    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxvzb\" (UniqueName: \"kubernetes.io/projected/34504b49-745e-4091-b543-01aed933d991-kube-api-access-gxvzb\") pod \"test-curl-7929-cb03b5\" (UID: \"34504b49-745e-4091-b543-01aed933d991\") " pod="opendatahub/test-curl-7929-cb03b5"
Sep 16 16:52:09.295942 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.295912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7929-cb03b5"
Sep 16 16:52:09.378186 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.378110    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:09.581635 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.581605    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7929-cb03b5"]
Sep 16 16:52:09.584320 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:09.584281    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34504b49_745e_4091_b543_01aed933d991.slice/crio-563f9c22aa5c6d0a5e75ed1bd7d26d7586c9fcda057f63bdf529ecd721773dc5 WatchSource:0}: Error finding container 563f9c22aa5c6d0a5e75ed1bd7d26d7586c9fcda057f63bdf529ecd721773dc5: Status 404 returned error can't find the container with id 563f9c22aa5c6d0a5e75ed1bd7d26d7586c9fcda057f63bdf529ecd721773dc5
Sep 16 16:52:09.626836 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.626806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:09.732508 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.732470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:09.820146 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.820092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:09.924817 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.924775    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"]
Sep 16 16:52:09.929015 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.928911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:09.931258 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.931234    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e2ea3e-dockercfg-q8h2c\""
Sep 16 16:52:09.931391 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.931263    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e2ea3e\""
Sep 16 16:52:09.939786 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:09.939758    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"]
Sep 16 16:52:10.032379 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.032347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42abc1d-e075-4181-aaf2-90ab21812e0c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs\" (UID: \"e42abc1d-e075-4181-aaf2-90ab21812e0c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:10.032540 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.032508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vlv6\" (UniqueName: \"kubernetes.io/projected/e42abc1d-e075-4181-aaf2-90ab21812e0c-kube-api-access-5vlv6\") pod \"payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs\" (UID: \"e42abc1d-e075-4181-aaf2-90ab21812e0c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:10.133735 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.133696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vlv6\" (UniqueName: \"kubernetes.io/projected/e42abc1d-e075-4181-aaf2-90ab21812e0c-kube-api-access-5vlv6\") pod \"payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs\" (UID: \"e42abc1d-e075-4181-aaf2-90ab21812e0c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:10.133928 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.133759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42abc1d-e075-4181-aaf2-90ab21812e0c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs\" (UID: \"e42abc1d-e075-4181-aaf2-90ab21812e0c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:10.134674 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.134645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42abc1d-e075-4181-aaf2-90ab21812e0c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs\" (UID: \"e42abc1d-e075-4181-aaf2-90ab21812e0c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:10.158860 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.158836    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vlv6\" (UniqueName: \"kubernetes.io/projected/e42abc1d-e075-4181-aaf2-90ab21812e0c-kube-api-access-5vlv6\") pod \"payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs\" (UID: \"e42abc1d-e075-4181-aaf2-90ab21812e0c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:10.179576 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.179486    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7929-cb03b5" event={"ID":"34504b49-745e-4091-b543-01aed933d991","Type":"ContainerStarted","Data":"8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd"}
Sep 16 16:52:10.179576 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.179571    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7929-cb03b5" event={"ID":"34504b49-745e-4091-b543-01aed933d991","Type":"ContainerStarted","Data":"563f9c22aa5c6d0a5e75ed1bd7d26d7586c9fcda057f63bdf529ecd721773dc5"}
Sep 16 16:52:10.179757 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.179666    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7929-98e07f" podUID="0fe6ad70-1027-476c-9144-b80dd3a25224" containerName="test-curl-7929-98e07f" containerID="cri-o://9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe" gracePeriod=2
Sep 16 16:52:10.207423 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.207376    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7929-cb03b5" podStartSLOduration=2.20736159 podStartE2EDuration="2.20736159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:10.205971934 +0000 UTC m=+945.702845058" watchObservedRunningTime="2026-09-16 16:52:10.20736159 +0000 UTC m=+945.704234714"
Sep 16 16:52:10.242978 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.242950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:10.390148 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.390068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:10.531775 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.531739    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"]
Sep 16 16:52:10.533396 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:10.533369    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42abc1d_e075_4181_aaf2_90ab21812e0c.slice/crio-9979fe1e745d4362801a704367a0f882f209932341fd0fb16f62f35c298d384c WatchSource:0}: Error finding container 9979fe1e745d4362801a704367a0f882f209932341fd0fb16f62f35c298d384c: Status 404 returned error can't find the container with id 9979fe1e745d4362801a704367a0f882f209932341fd0fb16f62f35c298d384c
Sep 16 16:52:10.834846 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.834753    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7929-cb03b5"]
Sep 16 16:52:10.841603 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:10.841577    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7929-cb03b5"]
Sep 16 16:52:11.187397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:11.187359    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs" event={"ID":"e42abc1d-e075-4181-aaf2-90ab21812e0c","Type":"ContainerStarted","Data":"341a093b856353bd27b5b5d2b03604795f44d1347cb7ecc665854cb60b78a78a"}
Sep 16 16:52:11.187618 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:11.187424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs" event={"ID":"e42abc1d-e075-4181-aaf2-90ab21812e0c","Type":"ContainerStarted","Data":"9979fe1e745d4362801a704367a0f882f209932341fd0fb16f62f35c298d384c"}
Sep 16 16:52:11.187618 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:11.187574    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:11.187765 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:52:11.187750    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7929-cb03b5" container="test-curl-7929-cb03b5"
Sep 16 16:52:11.217797 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:11.217742    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs" podStartSLOduration=2.217727778 podStartE2EDuration="2.217727778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:11.214919597 +0000 UTC m=+946.711792722" watchObservedRunningTime="2026-09-16 16:52:11.217727778 +0000 UTC m=+946.714600947"
Sep 16 16:52:11.286421 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:11.286373    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:11.382074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:11.382040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:11.627980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:11.627886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:11.831422 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:11.831382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:12.103773 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.103741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:12.193632 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.193597    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7929-cb03b5" podUID="34504b49-745e-4091-b543-01aed933d991" containerName="test-curl-7929-cb03b5" containerID="cri-o://8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd" gracePeriod=2
Sep 16 16:52:12.206056 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.206028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:12.310710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.310678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:12.369270 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.369213    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7929-98e07f"
Sep 16 16:52:12.456326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.456296    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hks78\" (UniqueName: \"kubernetes.io/projected/0fe6ad70-1027-476c-9144-b80dd3a25224-kube-api-access-hks78\") pod \"0fe6ad70-1027-476c-9144-b80dd3a25224\" (UID: \"0fe6ad70-1027-476c-9144-b80dd3a25224\") "
Sep 16 16:52:12.458927 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.458896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe6ad70-1027-476c-9144-b80dd3a25224-kube-api-access-hks78" (OuterVolumeSpecName: "kube-api-access-hks78") pod "0fe6ad70-1027-476c-9144-b80dd3a25224" (UID: "0fe6ad70-1027-476c-9144-b80dd3a25224"). InnerVolumeSpecName "kube-api-access-hks78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:52:12.557692 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.557658    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hks78\" (UniqueName: \"kubernetes.io/projected/0fe6ad70-1027-476c-9144-b80dd3a25224-kube-api-access-hks78\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:52:12.710781 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.710741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:12.967341 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.967240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:12.978034 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:12.978000    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fe6ad70-1027-476c-9144-b80dd3a25224" path="/var/lib/kubelet/pods/0fe6ad70-1027-476c-9144-b80dd3a25224/volumes"
Sep 16 16:52:13.202913 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.202877    2577 generic.go:356] "Generic (PLEG): container finished" podID="0fe6ad70-1027-476c-9144-b80dd3a25224" containerID="9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe" exitCode=137
Sep 16 16:52:13.203101 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.202934    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7929-98e07f"
Sep 16 16:52:13.203101 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.202954    2577 scope.go:117] "RemoveContainer" containerID="9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe"
Sep 16 16:52:13.215014 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.214982    2577 scope.go:117] "RemoveContainer" containerID="9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe"
Sep 16 16:52:13.215335 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:52:13.215312    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe\": container with ID starting with 9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe not found: ID does not exist" containerID="9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe"
Sep 16 16:52:13.215410 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.215348    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe"} err="failed to get container status \"9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe\": rpc error: code = NotFound desc = could not find container \"9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe\": container with ID starting with 9a201344c9c77490df52db8c75f1120df6341c1d22eb2e59ffb6e4653ba059fe not found: ID does not exist"
Sep 16 16:52:13.277919 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.277830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:13.359487 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.359450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:13.467064 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.467027    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:13.938070 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:13.938008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:14.032497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.032452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:14.087862 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.087821    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"]
Sep 16 16:52:14.088332 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.088312    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34504b49-745e-4091-b543-01aed933d991" containerName="test-curl-7929-cb03b5"
Sep 16 16:52:14.088332 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.088334    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="34504b49-745e-4091-b543-01aed933d991" containerName="test-curl-7929-cb03b5"
Sep 16 16:52:14.088439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.088402    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="34504b49-745e-4091-b543-01aed933d991" containerName="test-curl-7929-cb03b5"
Sep 16 16:52:14.092713 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.092693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:14.095216 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.095193    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-f604b4-dockercfg-42n72\""
Sep 16 16:52:14.095304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.095202    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-f604b4\""
Sep 16 16:52:14.103232 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.103207    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"]
Sep 16 16:52:14.151313 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.151278    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"]
Sep 16 16:52:14.154606 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.154589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:14.163967 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.163941    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"]
Sep 16 16:52:14.173285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.173257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n\" (UID: \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:14.173426 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.173310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8mng\" (UniqueName: \"kubernetes.io/projected/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-kube-api-access-n8mng\") pod \"payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n\" (UID: \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:14.275881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.274401    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7727f763-fd42-4c69-8564-ee3e007f69c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56\" (UID: \"7727f763-fd42-4c69-8564-ee3e007f69c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:14.275881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.274484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n\" (UID: \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:14.275881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.274549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8mng\" (UniqueName: \"kubernetes.io/projected/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-kube-api-access-n8mng\") pod \"payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n\" (UID: \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:14.275881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.274572    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kn9j\" (UniqueName: \"kubernetes.io/projected/7727f763-fd42-4c69-8564-ee3e007f69c4-kube-api-access-7kn9j\") pod \"payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56\" (UID: \"7727f763-fd42-4c69-8564-ee3e007f69c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:14.275881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.275426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n\" (UID: \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:14.307037 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.306992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8mng\" (UniqueName: \"kubernetes.io/projected/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-kube-api-access-n8mng\") pod \"payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n\" (UID: \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:14.379375 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.379238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kn9j\" (UniqueName: \"kubernetes.io/projected/7727f763-fd42-4c69-8564-ee3e007f69c4-kube-api-access-7kn9j\") pod \"payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56\" (UID: \"7727f763-fd42-4c69-8564-ee3e007f69c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:14.379501 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.379384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7727f763-fd42-4c69-8564-ee3e007f69c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56\" (UID: \"7727f763-fd42-4c69-8564-ee3e007f69c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:14.380360 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.380335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7727f763-fd42-4c69-8564-ee3e007f69c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56\" (UID: \"7727f763-fd42-4c69-8564-ee3e007f69c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:14.398318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.394358    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kn9j\" (UniqueName: \"kubernetes.io/projected/7727f763-fd42-4c69-8564-ee3e007f69c4-kube-api-access-7kn9j\") pod \"payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56\" (UID: \"7727f763-fd42-4c69-8564-ee3e007f69c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:14.399551 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.399401    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7929-cb03b5"
Sep 16 16:52:14.405480 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.405460    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:14.467845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.466103    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:14.567990 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.567913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:14.581857 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.581825    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxvzb\" (UniqueName: \"kubernetes.io/projected/34504b49-745e-4091-b543-01aed933d991-kube-api-access-gxvzb\") pod \"34504b49-745e-4091-b543-01aed933d991\" (UID: \"34504b49-745e-4091-b543-01aed933d991\") "
Sep 16 16:52:14.585292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.585250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34504b49-745e-4091-b543-01aed933d991-kube-api-access-gxvzb" (OuterVolumeSpecName: "kube-api-access-gxvzb") pod "34504b49-745e-4091-b543-01aed933d991" (UID: "34504b49-745e-4091-b543-01aed933d991"). InnerVolumeSpecName "kube-api-access-gxvzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:52:14.661770 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.661727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:14.682996 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.682965    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxvzb\" (UniqueName: \"kubernetes.io/projected/34504b49-745e-4091-b543-01aed933d991-kube-api-access-gxvzb\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:52:14.688129 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.688106    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"]
Sep 16 16:52:14.689610 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:14.689577    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4fdef40_5aa3_4be7_aaa9_46e83b717f30.slice/crio-ee8cb0c0cb63deb14dcd2819fb8ad361720a2fec90d24e6332bdfa85d1f03169 WatchSource:0}: Error finding container ee8cb0c0cb63deb14dcd2819fb8ad361720a2fec90d24e6332bdfa85d1f03169: Status 404 returned error can't find the container with id ee8cb0c0cb63deb14dcd2819fb8ad361720a2fec90d24e6332bdfa85d1f03169
Sep 16 16:52:14.753500 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.753478    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"]
Sep 16 16:52:14.757677 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:14.757652    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7727f763_fd42_4c69_8564_ee3e007f69c4.slice/crio-72e8a8902a77854e663217e733f9ed03d09e2bbf0f240e45527c7eaff338e529 WatchSource:0}: Error finding container 72e8a8902a77854e663217e733f9ed03d09e2bbf0f240e45527c7eaff338e529: Status 404 returned error can't find the container with id 72e8a8902a77854e663217e733f9ed03d09e2bbf0f240e45527c7eaff338e529
Sep 16 16:52:14.978452 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:14.978417    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34504b49-745e-4091-b543-01aed933d991" path="/var/lib/kubelet/pods/34504b49-745e-4091-b543-01aed933d991/volumes"
Sep 16 16:52:15.034388 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:52:15.034353    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34504b49_745e_4091_b543_01aed933d991.slice\": RecentStats: unable to find data in memory cache]"
Sep 16 16:52:15.222271 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.222227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n" event={"ID":"a4fdef40-5aa3-4be7-aaa9-46e83b717f30","Type":"ContainerStarted","Data":"456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3"}
Sep 16 16:52:15.222434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.222331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n" event={"ID":"a4fdef40-5aa3-4be7-aaa9-46e83b717f30","Type":"ContainerStarted","Data":"ee8cb0c0cb63deb14dcd2819fb8ad361720a2fec90d24e6332bdfa85d1f03169"}
Sep 16 16:52:15.222434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.222364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:15.223659 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.223629    2577 generic.go:356] "Generic (PLEG): container finished" podID="34504b49-745e-4091-b543-01aed933d991" containerID="8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd" exitCode=137
Sep 16 16:52:15.223759 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.223671    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7929-cb03b5"
Sep 16 16:52:15.223759 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.223723    2577 scope.go:117] "RemoveContainer" containerID="8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd"
Sep 16 16:52:15.226040 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.225687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56" event={"ID":"7727f763-fd42-4c69-8564-ee3e007f69c4","Type":"ContainerStarted","Data":"18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8"}
Sep 16 16:52:15.226040 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.225740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56" event={"ID":"7727f763-fd42-4c69-8564-ee3e007f69c4","Type":"ContainerStarted","Data":"72e8a8902a77854e663217e733f9ed03d09e2bbf0f240e45527c7eaff338e529"}
Sep 16 16:52:15.226040 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.225891    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:15.233955 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.233939    2577 scope.go:117] "RemoveContainer" containerID="8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd"
Sep 16 16:52:15.234235 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:52:15.234212    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd\": container with ID starting with 8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd not found: ID does not exist" containerID="8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd"
Sep 16 16:52:15.234293 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.234246    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd"} err="failed to get container status \"8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd\": rpc error: code = NotFound desc = could not find container \"8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd\": container with ID starting with 8f4bc93433fb9cfa81f7a028f3a3b885a46a8c5b02a53de8a7740fc00e315bbd not found: ID does not exist"
Sep 16 16:52:15.241192 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.241142    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n" podStartSLOduration=1.24113013 podStartE2EDuration="1.24113013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:15.240474049 +0000 UTC m=+950.737347175" watchObservedRunningTime="2026-09-16 16:52:15.24113013 +0000 UTC m=+950.738003253"
Sep 16 16:52:15.262056 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.261974    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56" podStartSLOduration=1.261963175 podStartE2EDuration="1.261963175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:15.259336711 +0000 UTC m=+950.756209835" watchObservedRunningTime="2026-09-16 16:52:15.261963175 +0000 UTC m=+950.758836299"
Sep 16 16:52:15.731673 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:15.731634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:16.047701 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:16.047614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:16.186375 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:16.186338    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:17.122140 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:17.122095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:18.092217 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:18.092181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:19.100183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.100137    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:19.170454 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.170423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:19.659253 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.659219    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:19.754132 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.754092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:19.802618 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.802584    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"]
Sep 16 16:52:19.807357 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.807338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:19.809932 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.809912    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-708c6a-dockercfg-tpq4v\""
Sep 16 16:52:19.810056 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.809945    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-708c6a\""
Sep 16 16:52:19.821403 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.821371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"]
Sep 16 16:52:19.838088 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.838057    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"]
Sep 16 16:52:19.841626 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.841610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:19.844642 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.844611    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:19.855722 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.855697    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"]
Sep 16 16:52:19.933983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.933877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddec1d46-4623-4d8d-a17d-3747b4ddef16-plugins-config-volume\") pod \"payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt\" (UID: \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\") " pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:19.933983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:19.933949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls7vp\" (UniqueName: \"kubernetes.io/projected/ddec1d46-4623-4d8d-a17d-3747b4ddef16-kube-api-access-ls7vp\") pod \"payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt\" (UID: \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\") " pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:20.034462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.034426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddec1d46-4623-4d8d-a17d-3747b4ddef16-plugins-config-volume\") pod \"payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt\" (UID: \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\") " pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:20.034648 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.034475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjc74\" (UniqueName: \"kubernetes.io/projected/e2265a70-f468-4d97-b29a-ba4423fabe6f-kube-api-access-gjc74\") pod \"payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2\" (UID: \"e2265a70-f468-4d97-b29a-ba4423fabe6f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:20.034648 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.034495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls7vp\" (UniqueName: \"kubernetes.io/projected/ddec1d46-4623-4d8d-a17d-3747b4ddef16-kube-api-access-ls7vp\") pod \"payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt\" (UID: \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\") " pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:20.034648 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.034609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2265a70-f468-4d97-b29a-ba4423fabe6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2\" (UID: \"e2265a70-f468-4d97-b29a-ba4423fabe6f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:20.035373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.035327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddec1d46-4623-4d8d-a17d-3747b4ddef16-plugins-config-volume\") pod \"payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt\" (UID: \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\") " pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:20.047287 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.047259    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls7vp\" (UniqueName: \"kubernetes.io/projected/ddec1d46-4623-4d8d-a17d-3747b4ddef16-kube-api-access-ls7vp\") pod \"payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt\" (UID: \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\") " pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:20.086573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.086513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:20.122276 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.122242    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:20.135725 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.135698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjc74\" (UniqueName: \"kubernetes.io/projected/e2265a70-f468-4d97-b29a-ba4423fabe6f-kube-api-access-gjc74\") pod \"payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2\" (UID: \"e2265a70-f468-4d97-b29a-ba4423fabe6f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:20.135813 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.135749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2265a70-f468-4d97-b29a-ba4423fabe6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2\" (UID: \"e2265a70-f468-4d97-b29a-ba4423fabe6f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:20.136397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.136379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2265a70-f468-4d97-b29a-ba4423fabe6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2\" (UID: \"e2265a70-f468-4d97-b29a-ba4423fabe6f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:20.144867 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.144847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjc74\" (UniqueName: \"kubernetes.io/projected/e2265a70-f468-4d97-b29a-ba4423fabe6f-kube-api-access-gjc74\") pod \"payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2\" (UID: \"e2265a70-f468-4d97-b29a-ba4423fabe6f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:20.151631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.151610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:20.156056 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.156034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:20.183019 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.182625    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:52:20.183019 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.182784    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:52:20.398106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.398079    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"]
Sep 16 16:52:20.399695 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:20.399663    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddec1d46_4623_4d8d_a17d_3747b4ddef16.slice/crio-17f0505d495d3dee4d6e5637c285f1ac41b5984c1b4a7ef61b15c9cd142ef962 WatchSource:0}: Error finding container 17f0505d495d3dee4d6e5637c285f1ac41b5984c1b4a7ef61b15c9cd142ef962: Status 404 returned error can't find the container with id 17f0505d495d3dee4d6e5637c285f1ac41b5984c1b4a7ef61b15c9cd142ef962
Sep 16 16:52:20.435272 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.435240    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"]
Sep 16 16:52:20.436328 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:20.436300    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2265a70_f468_4d97_b29a_ba4423fabe6f.slice/crio-d9078e95c7b4fff149f212c87c75127b1c09c4a01b71330626e6f61fa584365a WatchSource:0}: Error finding container d9078e95c7b4fff149f212c87c75127b1c09c4a01b71330626e6f61fa584365a: Status 404 returned error can't find the container with id d9078e95c7b4fff149f212c87c75127b1c09c4a01b71330626e6f61fa584365a
Sep 16 16:52:20.698271 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.698237    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:20.864166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.864084    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"]
Sep 16 16:52:20.867611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.867596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:20.871206 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.871184    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-41cc4972-dockercfg-wsbgd\""
Sep 16 16:52:20.871341 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.871236    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-41cc4972\""
Sep 16 16:52:20.893439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.893416    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"]
Sep 16 16:52:20.949380 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:20.949313    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:21.045217 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.045180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w6h5\" (UniqueName: \"kubernetes.io/projected/eb8e7023-5f12-45a0-a82c-ed3d98794b18-kube-api-access-5w6h5\") pod \"payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt\" (UID: \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\") " pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:21.045381 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.045249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8e7023-5f12-45a0-a82c-ed3d98794b18-plugins-config-volume\") pod \"payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt\" (UID: \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\") " pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:21.084368 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.084338    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:21.146467 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.146440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w6h5\" (UniqueName: \"kubernetes.io/projected/eb8e7023-5f12-45a0-a82c-ed3d98794b18-kube-api-access-5w6h5\") pod \"payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt\" (UID: \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\") " pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:21.146878 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.146510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8e7023-5f12-45a0-a82c-ed3d98794b18-plugins-config-volume\") pod \"payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt\" (UID: \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\") " pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:21.147190 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.147171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8e7023-5f12-45a0-a82c-ed3d98794b18-plugins-config-volume\") pod \"payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt\" (UID: \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\") " pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:21.162866 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.162839    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w6h5\" (UniqueName: \"kubernetes.io/projected/eb8e7023-5f12-45a0-a82c-ed3d98794b18-kube-api-access-5w6h5\") pod \"payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt\" (UID: \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\") " pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:21.178648 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.178629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:21.267172 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.267035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt" event={"ID":"ddec1d46-4623-4d8d-a17d-3747b4ddef16","Type":"ContainerStarted","Data":"b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363"}
Sep 16 16:52:21.267172 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.267117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt" event={"ID":"ddec1d46-4623-4d8d-a17d-3747b4ddef16","Type":"ContainerStarted","Data":"17f0505d495d3dee4d6e5637c285f1ac41b5984c1b4a7ef61b15c9cd142ef962"}
Sep 16 16:52:21.267172 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.267157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:21.269178 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.269144    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2" event={"ID":"e2265a70-f468-4d97-b29a-ba4423fabe6f","Type":"ContainerStarted","Data":"49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291"}
Sep 16 16:52:21.269304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.269193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2" event={"ID":"e2265a70-f468-4d97-b29a-ba4423fabe6f","Type":"ContainerStarted","Data":"d9078e95c7b4fff149f212c87c75127b1c09c4a01b71330626e6f61fa584365a"}
Sep 16 16:52:21.269304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.269287    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:21.299980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.299889    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt" podStartSLOduration=2.299870083 podStartE2EDuration="2.299870083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:21.297398713 +0000 UTC m=+956.794271838" watchObservedRunningTime="2026-09-16 16:52:21.299870083 +0000 UTC m=+956.796743209"
Sep 16 16:52:21.305655 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.305612    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:21.322857 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.322812    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2" podStartSLOduration=2.322798535 podStartE2EDuration="2.322798535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:21.320414974 +0000 UTC m=+956.817288113" watchObservedRunningTime="2026-09-16 16:52:21.322798535 +0000 UTC m=+956.819671658"
Sep 16 16:52:21.443553 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.443455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:21.471575 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.471552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"]
Sep 16 16:52:21.473078 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:21.473050    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8e7023_5f12_45a0_a82c_ed3d98794b18.slice/crio-066a0e10b93d0dd422c70d52a64f75a584d69dacf13fccd08cbae31c2e6bf0d5 WatchSource:0}: Error finding container 066a0e10b93d0dd422c70d52a64f75a584d69dacf13fccd08cbae31c2e6bf0d5: Status 404 returned error can't find the container with id 066a0e10b93d0dd422c70d52a64f75a584d69dacf13fccd08cbae31c2e6bf0d5
Sep 16 16:52:21.545261 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.545231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:21.688958 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.688924    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:21.906180 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:21.906138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:22.187722 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:22.187633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:22.196053 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:22.196028    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:52:22.279668 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:22.279579    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt" event={"ID":"eb8e7023-5f12-45a0-a82c-ed3d98794b18","Type":"ContainerStarted","Data":"daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6"}
Sep 16 16:52:22.279668 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:22.279643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt" event={"ID":"eb8e7023-5f12-45a0-a82c-ed3d98794b18","Type":"ContainerStarted","Data":"066a0e10b93d0dd422c70d52a64f75a584d69dacf13fccd08cbae31c2e6bf0d5"}
Sep 16 16:52:22.336302 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:22.336250    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt" podStartSLOduration=2.336230165 podStartE2EDuration="2.336230165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:22.33583592 +0000 UTC m=+957.832709045" watchObservedRunningTime="2026-09-16 16:52:22.336230165 +0000 UTC m=+957.833103288"
Sep 16 16:52:22.361447 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:22.361379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:22.654155 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:22.654113    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:22.837056 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:22.837014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:23.039764 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:23.039656    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:23.285374 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:23.285348    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:23.295695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:23.295607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:24.392100 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:24.392052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:25.204117 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:25.204082    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:25.338563 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:25.338444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:26.202880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:26.202837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:26.234362 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:26.234331    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:52:26.234519 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:26.234502    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:52:26.502429 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:26.502333    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:26.612008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:26.611958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:26.707546 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:26.707488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:26.804967 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:26.804872    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:27.081582 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:27.081488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:27.438613 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:27.438581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:28.486593 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:28.486557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:28.652122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:28.652063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:29.033783 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.033747    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:29.132884 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.132849    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:29.235995 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.235948    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"]
Sep 16 16:52:29.240591 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.240571    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:29.243165 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.243144    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ea8f09-dockercfg-fdxfk\""
Sep 16 16:52:29.243586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.243571    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ea8f09\""
Sep 16 16:52:29.250750 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.250728    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"]
Sep 16 16:52:29.287990 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.287928    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:29.319670 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.319647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szsgt\" (UniqueName: \"kubernetes.io/projected/34ad4da7-4a7f-432e-82ae-e9f4a995231e-kube-api-access-szsgt\") pod \"payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w\" (UID: \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:29.319792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.319757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad4da7-4a7f-432e-82ae-e9f4a995231e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w\" (UID: \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:29.420636 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.420598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szsgt\" (UniqueName: \"kubernetes.io/projected/34ad4da7-4a7f-432e-82ae-e9f4a995231e-kube-api-access-szsgt\") pod \"payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w\" (UID: \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:29.420800 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.420729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad4da7-4a7f-432e-82ae-e9f4a995231e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w\" (UID: \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:29.421419 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.421398    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad4da7-4a7f-432e-82ae-e9f4a995231e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w\" (UID: \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:29.429485 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.429461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szsgt\" (UniqueName: \"kubernetes.io/projected/34ad4da7-4a7f-432e-82ae-e9f4a995231e-kube-api-access-szsgt\") pod \"payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w\" (UID: \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:29.552124 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.552043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:29.584265 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.584230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:29.729362 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.729327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:29.815406 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.815330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:29.827560 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:29.827521    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"]
Sep 16 16:52:29.828971 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:29.828949    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ad4da7_4a7f_432e_82ae_e9f4a995231e.slice/crio-8b975263e97713058b285eb53378c8fcfd7662c40c480bf0c78e4c96e2f3377a WatchSource:0}: Error finding container 8b975263e97713058b285eb53378c8fcfd7662c40c480bf0c78e4c96e2f3377a: Status 404 returned error can't find the container with id 8b975263e97713058b285eb53378c8fcfd7662c40c480bf0c78e4c96e2f3377a
Sep 16 16:52:30.332557 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:30.332497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w" event={"ID":"34ad4da7-4a7f-432e-82ae-e9f4a995231e","Type":"ContainerStarted","Data":"51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271"}
Sep 16 16:52:30.332706 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:30.332586    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:30.332706 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:30.332610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w" event={"ID":"34ad4da7-4a7f-432e-82ae-e9f4a995231e","Type":"ContainerStarted","Data":"8b975263e97713058b285eb53378c8fcfd7662c40c480bf0c78e4c96e2f3377a"}
Sep 16 16:52:30.350700 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:30.350651    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w" podStartSLOduration=1.350636492 podStartE2EDuration="1.350636492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:30.34962733 +0000 UTC m=+965.846500458" watchObservedRunningTime="2026-09-16 16:52:30.350636492 +0000 UTC m=+965.847509652"
Sep 16 16:52:30.485634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:30.485605    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:31.346445 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:31.346399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:32.281294 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:32.281268    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:52:32.282151 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:32.282118    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:52:32.302802 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:32.302773    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:33.169223 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:33.169189    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:34.275015 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:34.274972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:34.293562 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:34.293518    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:34.398250 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:34.398216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:34.505044 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:34.505008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:34.599598 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:34.599490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:34.752681 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:34.752647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:35.399148 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:35.399106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:36.403873 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:36.403833    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:37.081096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:37.081063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:37.805956 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:37.805919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:38.521359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:38.521318    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:39.179339 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:39.179291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:39.330023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:39.329988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:40.298545 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:40.298487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:40.845899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:40.845859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:41.007555 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.007506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:41.153611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.153581    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"]
Sep 16 16:52:41.161242 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.161218    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"
Sep 16 16:52:41.163960 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.163938    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:52:41.164181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.164004    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:52:41.166809 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.166785    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"]
Sep 16 16:52:41.237778 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.237736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fec22f8-906d-4e53-8d32-3b08d04fbabe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr\" (UID: \"2fec22f8-906d-4e53-8d32-3b08d04fbabe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"
Sep 16 16:52:41.270832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.270799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:41.339362 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.339330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fec22f8-906d-4e53-8d32-3b08d04fbabe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr\" (UID: \"2fec22f8-906d-4e53-8d32-3b08d04fbabe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"
Sep 16 16:52:41.340101 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.340082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fec22f8-906d-4e53-8d32-3b08d04fbabe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr\" (UID: \"2fec22f8-906d-4e53-8d32-3b08d04fbabe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"
Sep 16 16:52:41.341285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.341266    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:52:41.474202 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.474130    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"
Sep 16 16:52:41.749869 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:41.749842    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"]
Sep 16 16:52:41.752244 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:41.752218    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fec22f8_906d_4e53_8d32_3b08d04fbabe.slice/crio-64db93e9d5f4b83991b37a0e3373be63dde23596c09afcd0b99dc7115969af9e WatchSource:0}: Error finding container 64db93e9d5f4b83991b37a0e3373be63dde23596c09afcd0b99dc7115969af9e: Status 404 returned error can't find the container with id 64db93e9d5f4b83991b37a0e3373be63dde23596c09afcd0b99dc7115969af9e
Sep 16 16:52:42.081481 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:42.081373    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:42.413767 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:42.413728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr" event={"ID":"2fec22f8-906d-4e53-8d32-3b08d04fbabe","Type":"ContainerStarted","Data":"64db93e9d5f4b83991b37a0e3373be63dde23596c09afcd0b99dc7115969af9e"}
Sep 16 16:52:42.904793 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:42.904760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:43.705461 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:43.705432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:44.396712 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:44.396667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:44.436089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:44.436058    2577 generic.go:356] "Generic (PLEG): container finished" podID="2fec22f8-906d-4e53-8d32-3b08d04fbabe" containerID="c3dee5e682bf6ebcd0d1a9fdee3bc016253416e617cad2897f60e0f127056be2" exitCode=0
Sep 16 16:52:44.436216 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:44.436097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr" event={"ID":"2fec22f8-906d-4e53-8d32-3b08d04fbabe","Type":"ContainerDied","Data":"c3dee5e682bf6ebcd0d1a9fdee3bc016253416e617cad2897f60e0f127056be2"}
Sep 16 16:52:45.073501 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:45.073464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:45.582432 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:45.582401    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"
Sep 16 16:52:45.682938 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:45.682903    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fec22f8-906d-4e53-8d32-3b08d04fbabe-maas-api-service-ca\") pod \"2fec22f8-906d-4e53-8d32-3b08d04fbabe\" (UID: \"2fec22f8-906d-4e53-8d32-3b08d04fbabe\") "
Sep 16 16:52:45.683228 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:45.683207    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fec22f8-906d-4e53-8d32-3b08d04fbabe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2fec22f8-906d-4e53-8d32-3b08d04fbabe" (UID: "2fec22f8-906d-4e53-8d32-3b08d04fbabe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:52:45.777425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:45.777330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:45.784461 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:45.784433    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fec22f8-906d-4e53-8d32-3b08d04fbabe-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:52:46.208872 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:46.208837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:46.449690 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:46.449661    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"
Sep 16 16:52:46.449852 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:46.449663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr" event={"ID":"2fec22f8-906d-4e53-8d32-3b08d04fbabe","Type":"ContainerDied","Data":"64db93e9d5f4b83991b37a0e3373be63dde23596c09afcd0b99dc7115969af9e"}
Sep 16 16:52:46.449852 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:46.449795    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64db93e9d5f4b83991b37a0e3373be63dde23596c09afcd0b99dc7115969af9e"
Sep 16 16:52:46.501909 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:46.501809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:47.449372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:47.449270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:48.077190 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.077154    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:48.119813 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.119782    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"]
Sep 16 16:52:48.120031 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.120001    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt" podUID="eb8e7023-5f12-45a0-a82c-ed3d98794b18" containerName="payload-processing" containerID="cri-o://daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6" gracePeriod=30
Sep 16 16:52:48.188492 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.188459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:48.331519 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.331442    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:48.377556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.377511    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:48.420200 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.420166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:48.442109 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.442080    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"]
Sep 16 16:52:48.444595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.444571    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-41cc4972-k6lfr"]
Sep 16 16:52:48.462850 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.462825    2577 generic.go:356] "Generic (PLEG): container finished" podID="eb8e7023-5f12-45a0-a82c-ed3d98794b18" containerID="daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6" exitCode=0
Sep 16 16:52:48.463158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.462875    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"
Sep 16 16:52:48.463158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.462910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt" event={"ID":"eb8e7023-5f12-45a0-a82c-ed3d98794b18","Type":"ContainerDied","Data":"daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6"}
Sep 16 16:52:48.463158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.462958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt" event={"ID":"eb8e7023-5f12-45a0-a82c-ed3d98794b18","Type":"ContainerDied","Data":"066a0e10b93d0dd422c70d52a64f75a584d69dacf13fccd08cbae31c2e6bf0d5"}
Sep 16 16:52:48.463158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.462981    2577 scope.go:117] "RemoveContainer" containerID="daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6"
Sep 16 16:52:48.479374 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.479349    2577 scope.go:117] "RemoveContainer" containerID="daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6"
Sep 16 16:52:48.479672 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:52:48.479652    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6\": container with ID starting with daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6 not found: ID does not exist" containerID="daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6"
Sep 16 16:52:48.479729 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.479681    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6"} err="failed to get container status \"daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6\": rpc error: code = NotFound desc = could not find container \"daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6\": container with ID starting with daba6fa3dfdd7a39b48386c173fe66d778f3608e3c966ed5392949f5f23d32a6 not found: ID does not exist"
Sep 16 16:52:48.486264 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.486238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:48.510960 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.510938    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8e7023-5f12-45a0-a82c-ed3d98794b18-plugins-config-volume\") pod \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\" (UID: \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\") "
Sep 16 16:52:48.511064 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.511005    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w6h5\" (UniqueName: \"kubernetes.io/projected/eb8e7023-5f12-45a0-a82c-ed3d98794b18-kube-api-access-5w6h5\") pod \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\" (UID: \"eb8e7023-5f12-45a0-a82c-ed3d98794b18\") "
Sep 16 16:52:48.511304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.511282    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb8e7023-5f12-45a0-a82c-ed3d98794b18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb8e7023-5f12-45a0-a82c-ed3d98794b18" (UID: "eb8e7023-5f12-45a0-a82c-ed3d98794b18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:52:48.513022 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.513002    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8e7023-5f12-45a0-a82c-ed3d98794b18-kube-api-access-5w6h5" (OuterVolumeSpecName: "kube-api-access-5w6h5") pod "eb8e7023-5f12-45a0-a82c-ed3d98794b18" (UID: "eb8e7023-5f12-45a0-a82c-ed3d98794b18"). InnerVolumeSpecName "kube-api-access-5w6h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:52:48.605992 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.605892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:48.611980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.611952    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb8e7023-5f12-45a0-a82c-ed3d98794b18-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:52:48.611980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.611979    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w6h5\" (UniqueName: \"kubernetes.io/projected/eb8e7023-5f12-45a0-a82c-ed3d98794b18-kube-api-access-5w6h5\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:52:48.788595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.788564    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"]
Sep 16 16:52:48.792624 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.792596    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-41cc4972-54984bfb89-qd4wt"]
Sep 16 16:52:48.919342 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.919294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:48.979450 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.979418    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fec22f8-906d-4e53-8d32-3b08d04fbabe" path="/var/lib/kubelet/pods/2fec22f8-906d-4e53-8d32-3b08d04fbabe/volumes"
Sep 16 16:52:48.979828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:48.979815    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb8e7023-5f12-45a0-a82c-ed3d98794b18" path="/var/lib/kubelet/pods/eb8e7023-5f12-45a0-a82c-ed3d98794b18/volumes"
Sep 16 16:52:49.114869 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:49.114817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:49.265380 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:49.265264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:49.359483 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:49.359445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:49.570500 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:49.570405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:49.886523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:49.886406    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:50.393367 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:50.393334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:50.806092 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:50.806002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:51.696242 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:51.696207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:52.371176 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:52.371141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:52.472716 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:52.472672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:53.181334 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:53.181298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:54.052192 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.052151    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:54.152836 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.152787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:54.516118 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.516085    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:54.609997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.609960    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:54.635038 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.635002    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"]
Sep 16 16:52:54.635492 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.635475    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb8e7023-5f12-45a0-a82c-ed3d98794b18" containerName="payload-processing"
Sep 16 16:52:54.635492 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.635495    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb8e7023-5f12-45a0-a82c-ed3d98794b18" containerName="payload-processing"
Sep 16 16:52:54.635683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.635553    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fec22f8-906d-4e53-8d32-3b08d04fbabe" containerName="revoke-keys"
Sep 16 16:52:54.635683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.635563    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fec22f8-906d-4e53-8d32-3b08d04fbabe" containerName="revoke-keys"
Sep 16 16:52:54.635683 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.635672    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb8e7023-5f12-45a0-a82c-ed3d98794b18" containerName="payload-processing"
Sep 16 16:52:54.635829 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.635685    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fec22f8-906d-4e53-8d32-3b08d04fbabe" containerName="revoke-keys"
Sep 16 16:52:54.638832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.638812    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:54.641500 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.641481    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-241a48eb-dockercfg-tr78g\""
Sep 16 16:52:54.641621 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.641567    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-241a48eb\""
Sep 16 16:52:54.653216 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.653191    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"]
Sep 16 16:52:54.673084 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.673056    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"]
Sep 16 16:52:54.676570 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.676553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:54.691922 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.691893    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"]
Sep 16 16:52:54.693877 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.693856    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:54.769308 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.769236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w48t\" (UniqueName: \"kubernetes.io/projected/84635f97-4e6b-4de0-86d1-b16948665d32-kube-api-access-8w48t\") pod \"payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw\" (UID: \"84635f97-4e6b-4de0-86d1-b16948665d32\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:54.769308 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.769279    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38db9ee3-4576-48d4-90c0-aa8ee06a8277-plugins-config-volume\") pod \"payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9\" (UID: \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\") " pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:54.769483 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.769342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84635f97-4e6b-4de0-86d1-b16948665d32-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw\" (UID: \"84635f97-4e6b-4de0-86d1-b16948665d32\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:54.769483 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.769378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6g7g\" (UniqueName: \"kubernetes.io/projected/38db9ee3-4576-48d4-90c0-aa8ee06a8277-kube-api-access-z6g7g\") pod \"payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9\" (UID: \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\") " pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:54.870483 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.870448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w48t\" (UniqueName: \"kubernetes.io/projected/84635f97-4e6b-4de0-86d1-b16948665d32-kube-api-access-8w48t\") pod \"payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw\" (UID: \"84635f97-4e6b-4de0-86d1-b16948665d32\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:54.870684 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.870493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38db9ee3-4576-48d4-90c0-aa8ee06a8277-plugins-config-volume\") pod \"payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9\" (UID: \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\") " pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:54.870684 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.870520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84635f97-4e6b-4de0-86d1-b16948665d32-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw\" (UID: \"84635f97-4e6b-4de0-86d1-b16948665d32\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:54.870684 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.870569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6g7g\" (UniqueName: \"kubernetes.io/projected/38db9ee3-4576-48d4-90c0-aa8ee06a8277-kube-api-access-z6g7g\") pod \"payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9\" (UID: \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\") " pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:54.871300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.871276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38db9ee3-4576-48d4-90c0-aa8ee06a8277-plugins-config-volume\") pod \"payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9\" (UID: \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\") " pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:54.871300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.871288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84635f97-4e6b-4de0-86d1-b16948665d32-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw\" (UID: \"84635f97-4e6b-4de0-86d1-b16948665d32\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:54.879424 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.879401    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w48t\" (UniqueName: \"kubernetes.io/projected/84635f97-4e6b-4de0-86d1-b16948665d32-kube-api-access-8w48t\") pod \"payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw\" (UID: \"84635f97-4e6b-4de0-86d1-b16948665d32\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:54.879556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.879510    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6g7g\" (UniqueName: \"kubernetes.io/projected/38db9ee3-4576-48d4-90c0-aa8ee06a8277-kube-api-access-z6g7g\") pod \"payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9\" (UID: \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\") " pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:54.939784 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.939761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:54.948560 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.948521    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:54.986850 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:54.986823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:55.230901 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.230873    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"]
Sep 16 16:52:55.231842 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:55.231822    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84635f97_4e6b_4de0_86d1_b16948665d32.slice/crio-4f963d62b7f37dbe6516cf317f016d39af3dd782f3470db7984e9332d80e66e9 WatchSource:0}: Error finding container 4f963d62b7f37dbe6516cf317f016d39af3dd782f3470db7984e9332d80e66e9: Status 404 returned error can't find the container with id 4f963d62b7f37dbe6516cf317f016d39af3dd782f3470db7984e9332d80e66e9
Sep 16 16:52:55.274246 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:52:55.274197    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38db9ee3_4576_48d4_90c0_aa8ee06a8277.slice/crio-4f4a7c8ff262c15334e94f73651b315e6b8b116379c68b1705f86652c928a65e WatchSource:0}: Error finding container 4f4a7c8ff262c15334e94f73651b315e6b8b116379c68b1705f86652c928a65e: Status 404 returned error can't find the container with id 4f4a7c8ff262c15334e94f73651b315e6b8b116379c68b1705f86652c928a65e
Sep 16 16:52:55.281586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.275140    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"]
Sep 16 16:52:55.511063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.511024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9" event={"ID":"38db9ee3-4576-48d4-90c0-aa8ee06a8277","Type":"ContainerStarted","Data":"2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15"}
Sep 16 16:52:55.511243 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.511080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9" event={"ID":"38db9ee3-4576-48d4-90c0-aa8ee06a8277","Type":"ContainerStarted","Data":"4f4a7c8ff262c15334e94f73651b315e6b8b116379c68b1705f86652c928a65e"}
Sep 16 16:52:55.511243 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.511122    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:52:55.512899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.512868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw" event={"ID":"84635f97-4e6b-4de0-86d1-b16948665d32","Type":"ContainerStarted","Data":"08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e"}
Sep 16 16:52:55.513006 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.512923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw" event={"ID":"84635f97-4e6b-4de0-86d1-b16948665d32","Type":"ContainerStarted","Data":"4f963d62b7f37dbe6516cf317f016d39af3dd782f3470db7984e9332d80e66e9"}
Sep 16 16:52:55.513006 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.512947    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:52:55.529699 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.529657    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9" podStartSLOduration=1.529643574 podStartE2EDuration="1.529643574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:55.526085529 +0000 UTC m=+991.022958658" watchObservedRunningTime="2026-09-16 16:52:55.529643574 +0000 UTC m=+991.026516697"
Sep 16 16:52:55.542832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.542787    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw" podStartSLOduration=1.542778442 podStartE2EDuration="1.542778442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:52:55.54162447 +0000 UTC m=+991.038497593" watchObservedRunningTime="2026-09-16 16:52:55.542778442 +0000 UTC m=+991.039651603"
Sep 16 16:52:55.796972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:55.796934    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:56.629059 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:56.629022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:52:57.318613 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:52:57.318575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:06.522237 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:06.522208    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:53:06.522660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:06.522278    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:53:16.403819 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:16.403771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:17.342507 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.342472    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"]
Sep 16 16:53:17.350262 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.350241    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"
Sep 16 16:53:17.352712 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.352686    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:53:17.352899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.352774    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:53:17.355701 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.355673    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"]
Sep 16 16:53:17.397886 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.397860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:17.473728 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.473690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95423a60-ddb7-412e-b2e8-f4c304950bf8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q\" (UID: \"95423a60-ddb7-412e-b2e8-f4c304950bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"
Sep 16 16:53:17.574279 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.574248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95423a60-ddb7-412e-b2e8-f4c304950bf8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q\" (UID: \"95423a60-ddb7-412e-b2e8-f4c304950bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"
Sep 16 16:53:17.574906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.574886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95423a60-ddb7-412e-b2e8-f4c304950bf8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q\" (UID: \"95423a60-ddb7-412e-b2e8-f4c304950bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"
Sep 16 16:53:17.663078 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.663051    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"
Sep 16 16:53:17.947439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:17.947412    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"]
Sep 16 16:53:17.948142 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:53:17.948111    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95423a60_ddb7_412e_b2e8_f4c304950bf8.slice/crio-9312a9fedb632a3584a05f87ef395fa9cc452573bd44bc9821bb3fe11a47df6b WatchSource:0}: Error finding container 9312a9fedb632a3584a05f87ef395fa9cc452573bd44bc9821bb3fe11a47df6b: Status 404 returned error can't find the container with id 9312a9fedb632a3584a05f87ef395fa9cc452573bd44bc9821bb3fe11a47df6b
Sep 16 16:53:18.210994 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:18.210966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:18.652682 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:18.652639    2577 generic.go:356] "Generic (PLEG): container finished" podID="95423a60-ddb7-412e-b2e8-f4c304950bf8" containerID="771f7fb59c51d234d24460bd1b4485f8db6ebd0b739f117bc205bbabb069c90f" exitCode=0
Sep 16 16:53:18.653063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:18.652727    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q" event={"ID":"95423a60-ddb7-412e-b2e8-f4c304950bf8","Type":"ContainerDied","Data":"771f7fb59c51d234d24460bd1b4485f8db6ebd0b739f117bc205bbabb069c90f"}
Sep 16 16:53:18.653063 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:18.652779    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q" event={"ID":"95423a60-ddb7-412e-b2e8-f4c304950bf8","Type":"ContainerStarted","Data":"9312a9fedb632a3584a05f87ef395fa9cc452573bd44bc9821bb3fe11a47df6b"}
Sep 16 16:53:19.030834 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:19.030791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:19.802703 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:19.802670    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"
Sep 16 16:53:19.897100 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:19.897059    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95423a60-ddb7-412e-b2e8-f4c304950bf8-maas-api-service-ca\") pod \"95423a60-ddb7-412e-b2e8-f4c304950bf8\" (UID: \"95423a60-ddb7-412e-b2e8-f4c304950bf8\") "
Sep 16 16:53:19.897408 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:19.897385    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95423a60-ddb7-412e-b2e8-f4c304950bf8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95423a60-ddb7-412e-b2e8-f4c304950bf8" (UID: "95423a60-ddb7-412e-b2e8-f4c304950bf8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:53:19.998599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:19.998493    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95423a60-ddb7-412e-b2e8-f4c304950bf8-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:53:20.095597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:20.095510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:20.666344 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:20.666311    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"
Sep 16 16:53:20.666517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:20.666342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q" event={"ID":"95423a60-ddb7-412e-b2e8-f4c304950bf8","Type":"ContainerDied","Data":"9312a9fedb632a3584a05f87ef395fa9cc452573bd44bc9821bb3fe11a47df6b"}
Sep 16 16:53:20.666517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:20.666395    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9312a9fedb632a3584a05f87ef395fa9cc452573bd44bc9821bb3fe11a47df6b"
Sep 16 16:53:21.059005 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:21.058899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:21.904978 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:21.904940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:22.847209 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:22.847144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:23.949913 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:23.949862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:24.581599 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.581557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:24.643741 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.643708    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"]
Sep 16 16:53:24.643984 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.643959    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9" podUID="38db9ee3-4576-48d4-90c0-aa8ee06a8277" containerName="payload-processing" containerID="cri-o://2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15" gracePeriod=30
Sep 16 16:53:24.654439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.653906    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"]
Sep 16 16:53:24.654439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.654133    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw" podUID="84635f97-4e6b-4de0-86d1-b16948665d32" containerName="payload-pre-processing" containerID="cri-o://08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e" gracePeriod=30
Sep 16 16:53:24.697930 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.697896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:24.795159 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.795123    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:24.902862 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.902816    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:24.913905 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.913863    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"]
Sep 16 16:53:24.917280 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.917224    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-241a48eb-hkw7q"]
Sep 16 16:53:24.940655 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.940636    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:53:24.944383 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.944365    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:53:24.979377 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:24.979357    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95423a60-ddb7-412e-b2e8-f4c304950bf8" path="/var/lib/kubelet/pods/95423a60-ddb7-412e-b2e8-f4c304950bf8/volumes"
Sep 16 16:53:25.004742 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.004697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:25.046164 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.046133    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38db9ee3-4576-48d4-90c0-aa8ee06a8277-plugins-config-volume\") pod \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\" (UID: \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\") "
Sep 16 16:53:25.046325 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.046204    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84635f97-4e6b-4de0-86d1-b16948665d32-plugins-config-volume\") pod \"84635f97-4e6b-4de0-86d1-b16948665d32\" (UID: \"84635f97-4e6b-4de0-86d1-b16948665d32\") "
Sep 16 16:53:25.046325 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.046235    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w48t\" (UniqueName: \"kubernetes.io/projected/84635f97-4e6b-4de0-86d1-b16948665d32-kube-api-access-8w48t\") pod \"84635f97-4e6b-4de0-86d1-b16948665d32\" (UID: \"84635f97-4e6b-4de0-86d1-b16948665d32\") "
Sep 16 16:53:25.046325 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.046293    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6g7g\" (UniqueName: \"kubernetes.io/projected/38db9ee3-4576-48d4-90c0-aa8ee06a8277-kube-api-access-z6g7g\") pod \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\" (UID: \"38db9ee3-4576-48d4-90c0-aa8ee06a8277\") "
Sep 16 16:53:25.046556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.046514    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38db9ee3-4576-48d4-90c0-aa8ee06a8277-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38db9ee3-4576-48d4-90c0-aa8ee06a8277" (UID: "38db9ee3-4576-48d4-90c0-aa8ee06a8277"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:53:25.046637 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.046621    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84635f97-4e6b-4de0-86d1-b16948665d32-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84635f97-4e6b-4de0-86d1-b16948665d32" (UID: "84635f97-4e6b-4de0-86d1-b16948665d32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:53:25.046697 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.046635    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38db9ee3-4576-48d4-90c0-aa8ee06a8277-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:53:25.048470 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.048442    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84635f97-4e6b-4de0-86d1-b16948665d32-kube-api-access-8w48t" (OuterVolumeSpecName: "kube-api-access-8w48t") pod "84635f97-4e6b-4de0-86d1-b16948665d32" (UID: "84635f97-4e6b-4de0-86d1-b16948665d32"). InnerVolumeSpecName "kube-api-access-8w48t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:53:25.048750 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.048623    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38db9ee3-4576-48d4-90c0-aa8ee06a8277-kube-api-access-z6g7g" (OuterVolumeSpecName: "kube-api-access-z6g7g") pod "38db9ee3-4576-48d4-90c0-aa8ee06a8277" (UID: "38db9ee3-4576-48d4-90c0-aa8ee06a8277"). InnerVolumeSpecName "kube-api-access-z6g7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:53:25.094689 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.094659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:25.147181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.147152    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84635f97-4e6b-4de0-86d1-b16948665d32-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:53:25.147181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.147182    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w48t\" (UniqueName: \"kubernetes.io/projected/84635f97-4e6b-4de0-86d1-b16948665d32-kube-api-access-8w48t\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:53:25.147350 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.147197    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6g7g\" (UniqueName: \"kubernetes.io/projected/38db9ee3-4576-48d4-90c0-aa8ee06a8277-kube-api-access-z6g7g\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:53:25.434784 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.434684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:25.546556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.546500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:25.698815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.698742    2577 generic.go:356] "Generic (PLEG): container finished" podID="38db9ee3-4576-48d4-90c0-aa8ee06a8277" containerID="2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15" exitCode=0
Sep 16 16:53:25.698815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.698800    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"
Sep 16 16:53:25.698986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.698828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9" event={"ID":"38db9ee3-4576-48d4-90c0-aa8ee06a8277","Type":"ContainerDied","Data":"2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15"}
Sep 16 16:53:25.698986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.698917    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9" event={"ID":"38db9ee3-4576-48d4-90c0-aa8ee06a8277","Type":"ContainerDied","Data":"4f4a7c8ff262c15334e94f73651b315e6b8b116379c68b1705f86652c928a65e"}
Sep 16 16:53:25.698986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.698945    2577 scope.go:117] "RemoveContainer" containerID="2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15"
Sep 16 16:53:25.700339 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.700131    2577 generic.go:356] "Generic (PLEG): container finished" podID="84635f97-4e6b-4de0-86d1-b16948665d32" containerID="08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e" exitCode=0
Sep 16 16:53:25.700339 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.700164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw" event={"ID":"84635f97-4e6b-4de0-86d1-b16948665d32","Type":"ContainerDied","Data":"08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e"}
Sep 16 16:53:25.700339 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.700186    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"
Sep 16 16:53:25.700339 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.700206    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw" event={"ID":"84635f97-4e6b-4de0-86d1-b16948665d32","Type":"ContainerDied","Data":"4f963d62b7f37dbe6516cf317f016d39af3dd782f3470db7984e9332d80e66e9"}
Sep 16 16:53:25.709062 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.709040    2577 scope.go:117] "RemoveContainer" containerID="2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15"
Sep 16 16:53:25.709381 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:53:25.709359    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15\": container with ID starting with 2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15 not found: ID does not exist" containerID="2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15"
Sep 16 16:53:25.709476 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.709387    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15"} err="failed to get container status \"2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15\": rpc error: code = NotFound desc = could not find container \"2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15\": container with ID starting with 2df44a296779ddd0689f2f042494a8b4109e53e3f664aad225e255c313b5ac15 not found: ID does not exist"
Sep 16 16:53:25.709476 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.709404    2577 scope.go:117] "RemoveContainer" containerID="08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e"
Sep 16 16:53:25.717743 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.717726    2577 scope.go:117] "RemoveContainer" containerID="08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e"
Sep 16 16:53:25.717993 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:53:25.717977    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e\": container with ID starting with 08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e not found: ID does not exist" containerID="08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e"
Sep 16 16:53:25.718056 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.718003    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e"} err="failed to get container status \"08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e\": rpc error: code = NotFound desc = could not find container \"08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e\": container with ID starting with 08fe007a0ca3faadfb1961d2df594807f54c805cc7b98e4fcd44016b55f5579e not found: ID does not exist"
Sep 16 16:53:25.737057 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.737033    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"]
Sep 16 16:53:25.750490 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.750464    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-241a48eb-5fb95f6dc8-k9ll9"]
Sep 16 16:53:25.780614 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.780584    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"]
Sep 16 16:53:25.785337 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.785310    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-241a48eb-85b46f7d8b-f4fvw"]
Sep 16 16:53:25.895940 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:25.895910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:26.005736 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:26.005638    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:26.488842 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:26.488801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:26.978142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:26.978113    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38db9ee3-4576-48d4-90c0-aa8ee06a8277" path="/var/lib/kubelet/pods/38db9ee3-4576-48d4-90c0-aa8ee06a8277/volumes"
Sep 16 16:53:26.978559 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:26.978543    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84635f97-4e6b-4de0-86d1-b16948665d32" path="/var/lib/kubelet/pods/84635f97-4e6b-4de0-86d1-b16948665d32/volumes"
Sep 16 16:53:27.233139 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:27.233030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:28.526328 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:28.526291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:28.671066 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:28.671033    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:28.782847 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:28.782755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:29.657457 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:29.657420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:30.797481 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:30.797437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:31.828482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:31.828439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:31.918814 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:31.918773    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:32.163395 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.163352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:32.504933 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.504819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:32.618305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.618256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:32.709639 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.709611    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"]
Sep 16 16:53:32.710029 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710018    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38db9ee3-4576-48d4-90c0-aa8ee06a8277" containerName="payload-processing"
Sep 16 16:53:32.710077 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710031    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38db9ee3-4576-48d4-90c0-aa8ee06a8277" containerName="payload-processing"
Sep 16 16:53:32.710077 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710047    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84635f97-4e6b-4de0-86d1-b16948665d32" containerName="payload-pre-processing"
Sep 16 16:53:32.710077 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710052    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="84635f97-4e6b-4de0-86d1-b16948665d32" containerName="payload-pre-processing"
Sep 16 16:53:32.710077 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710070    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95423a60-ddb7-412e-b2e8-f4c304950bf8" containerName="revoke-keys"
Sep 16 16:53:32.710077 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710076    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95423a60-ddb7-412e-b2e8-f4c304950bf8" containerName="revoke-keys"
Sep 16 16:53:32.710229 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710146    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="84635f97-4e6b-4de0-86d1-b16948665d32" containerName="payload-pre-processing"
Sep 16 16:53:32.710229 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710158    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38db9ee3-4576-48d4-90c0-aa8ee06a8277" containerName="payload-processing"
Sep 16 16:53:32.710229 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.710165    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95423a60-ddb7-412e-b2e8-f4c304950bf8" containerName="revoke-keys"
Sep 16 16:53:32.714934 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.714916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:32.718327 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.718305    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1fd647e0-dockercfg-szgsw\""
Sep 16 16:53:32.718932 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.718914    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1fd647e0\""
Sep 16 16:53:32.727153 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.727132    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"]
Sep 16 16:53:32.802278 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.802213    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"]
Sep 16 16:53:32.805710 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.805693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:32.817474 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.817452    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"]
Sep 16 16:53:32.824343 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.824316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l\" (UID: \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:32.824343 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.824345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82ppz\" (UniqueName: \"kubernetes.io/projected/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-kube-api-access-82ppz\") pod \"payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l\" (UID: \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:32.901708 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.901673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:32.925655 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.925623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l\" (UID: \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:32.925792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.925658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82ppz\" (UniqueName: \"kubernetes.io/projected/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-kube-api-access-82ppz\") pod \"payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l\" (UID: \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:32.925860 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.925790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5655cc5-a18e-4636-9523-e6293c18627c-plugins-config-volume\") pod \"payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s\" (UID: \"f5655cc5-a18e-4636-9523-e6293c18627c\") " pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:32.925860 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.925837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs66q\" (UniqueName: \"kubernetes.io/projected/f5655cc5-a18e-4636-9523-e6293c18627c-kube-api-access-bs66q\") pod \"payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s\" (UID: \"f5655cc5-a18e-4636-9523-e6293c18627c\") " pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:32.926360 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.926338    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l\" (UID: \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:32.937222 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:32.937197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82ppz\" (UniqueName: \"kubernetes.io/projected/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-kube-api-access-82ppz\") pod \"payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l\" (UID: \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:33.025728 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.025700    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:33.027100 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.027071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5655cc5-a18e-4636-9523-e6293c18627c-plugins-config-volume\") pod \"payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s\" (UID: \"f5655cc5-a18e-4636-9523-e6293c18627c\") " pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:33.027216 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.027110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs66q\" (UniqueName: \"kubernetes.io/projected/f5655cc5-a18e-4636-9523-e6293c18627c-kube-api-access-bs66q\") pod \"payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s\" (UID: \"f5655cc5-a18e-4636-9523-e6293c18627c\") " pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:33.027729 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.027709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5655cc5-a18e-4636-9523-e6293c18627c-plugins-config-volume\") pod \"payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s\" (UID: \"f5655cc5-a18e-4636-9523-e6293c18627c\") " pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:33.039941 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.039921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs66q\" (UniqueName: \"kubernetes.io/projected/f5655cc5-a18e-4636-9523-e6293c18627c-kube-api-access-bs66q\") pod \"payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s\" (UID: \"f5655cc5-a18e-4636-9523-e6293c18627c\") " pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:33.065642 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.065548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:33.115980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.115951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:33.321064 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:53:33.321034    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed33ca0_b2ea_4ca3_8813_655c6152f56d.slice/crio-e96bb8cf9a5b6039d594a7893d2253b645d28c847e62131e2c65fb2d0371b417 WatchSource:0}: Error finding container e96bb8cf9a5b6039d594a7893d2253b645d28c847e62131e2c65fb2d0371b417: Status 404 returned error can't find the container with id e96bb8cf9a5b6039d594a7893d2253b645d28c847e62131e2c65fb2d0371b417
Sep 16 16:53:33.325481 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.325461    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"]
Sep 16 16:53:33.432457 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.432424    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"]
Sep 16 16:53:33.436485 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:53:33.436458    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5655cc5_a18e_4636_9523_e6293c18627c.slice/crio-be54770a2b0dcbe1edb211da40d8bb0080b8273995ebbed868da9d719f5abfca WatchSource:0}: Error finding container be54770a2b0dcbe1edb211da40d8bb0080b8273995ebbed868da9d719f5abfca: Status 404 returned error can't find the container with id be54770a2b0dcbe1edb211da40d8bb0080b8273995ebbed868da9d719f5abfca
Sep 16 16:53:33.756324 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.756288    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s" event={"ID":"f5655cc5-a18e-4636-9523-e6293c18627c","Type":"ContainerStarted","Data":"dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1"}
Sep 16 16:53:33.756586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.756353    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s" event={"ID":"f5655cc5-a18e-4636-9523-e6293c18627c","Type":"ContainerStarted","Data":"be54770a2b0dcbe1edb211da40d8bb0080b8273995ebbed868da9d719f5abfca"}
Sep 16 16:53:33.756586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.756484    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:33.757973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.757948    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l" event={"ID":"5ed33ca0-b2ea-4ca3-8813-655c6152f56d","Type":"ContainerStarted","Data":"3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6"}
Sep 16 16:53:33.758078 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.757986    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l" event={"ID":"5ed33ca0-b2ea-4ca3-8813-655c6152f56d","Type":"ContainerStarted","Data":"e96bb8cf9a5b6039d594a7893d2253b645d28c847e62131e2c65fb2d0371b417"}
Sep 16 16:53:33.758078 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.758006    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:33.777148 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.777099    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s" podStartSLOduration=1.7770857549999999 podStartE2EDuration="1.777085755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:53:33.775881819 +0000 UTC m=+1029.272754942" watchObservedRunningTime="2026-09-16 16:53:33.777085755 +0000 UTC m=+1029.273958878"
Sep 16 16:53:33.794582 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:33.794502    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l" podStartSLOduration=1.794485012 podStartE2EDuration="1.794485012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:53:33.794447213 +0000 UTC m=+1029.291320338" watchObservedRunningTime="2026-09-16 16:53:33.794485012 +0000 UTC m=+1029.291358138"
Sep 16 16:53:34.115074 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:34.114969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:35.008294 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:35.008255    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:35.977330 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:35.977297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:37.525813 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:37.525767    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:44.766404 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:44.766375    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:53:44.766824 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:44.766561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:53:54.283653 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:54.283616    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:54.981335 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:54.981287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:55.697713 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:55.697673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:56.601122 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:56.601088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:57.354415 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:57.354376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:58.185158 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:58.185126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:59.132334 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.132306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:53:59.364292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.364255    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"]
Sep 16 16:53:59.367926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.367898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"
Sep 16 16:53:59.370449 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.370431    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:53:59.370542 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.370438    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:53:59.380260 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.380237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"]
Sep 16 16:53:59.459942 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.459911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c387d6ab-0eb3-4477-81e6-af5fdc86d18a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1fd647e0-k7844\" (UID: \"c387d6ab-0eb3-4477-81e6-af5fdc86d18a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"
Sep 16 16:53:59.560980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.560945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c387d6ab-0eb3-4477-81e6-af5fdc86d18a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1fd647e0-k7844\" (UID: \"c387d6ab-0eb3-4477-81e6-af5fdc86d18a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"
Sep 16 16:53:59.561625 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.561602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c387d6ab-0eb3-4477-81e6-af5fdc86d18a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1fd647e0-k7844\" (UID: \"c387d6ab-0eb3-4477-81e6-af5fdc86d18a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"
Sep 16 16:53:59.679270 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.679232    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"
Sep 16 16:53:59.952234 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:53:59.952201    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"]
Sep 16 16:53:59.954677 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:53:59.954648    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc387d6ab_0eb3_4477_81e6_af5fdc86d18a.slice/crio-d50a348a5db398d451b707c1a6cfdc704d02ae030564408a3fa31d721071e6ca WatchSource:0}: Error finding container d50a348a5db398d451b707c1a6cfdc704d02ae030564408a3fa31d721071e6ca: Status 404 returned error can't find the container with id d50a348a5db398d451b707c1a6cfdc704d02ae030564408a3fa31d721071e6ca
Sep 16 16:54:00.106916 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:00.106883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:00.821648 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:00.821602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:00.928977 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:00.928944    2577 generic.go:356] "Generic (PLEG): container finished" podID="c387d6ab-0eb3-4477-81e6-af5fdc86d18a" containerID="0ff3c728319ae4e2592210bd858af2d72f83fb433d113c09b5f40577a447367f" exitCode=0
Sep 16 16:54:00.929136 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:00.929029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844" event={"ID":"c387d6ab-0eb3-4477-81e6-af5fdc86d18a","Type":"ContainerDied","Data":"0ff3c728319ae4e2592210bd858af2d72f83fb433d113c09b5f40577a447367f"}
Sep 16 16:54:00.929136 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:00.929083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844" event={"ID":"c387d6ab-0eb3-4477-81e6-af5fdc86d18a","Type":"ContainerStarted","Data":"d50a348a5db398d451b707c1a6cfdc704d02ae030564408a3fa31d721071e6ca"}
Sep 16 16:54:01.577720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:01.577685    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:02.074909 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:02.074887    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"
Sep 16 16:54:02.186285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:02.186193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c387d6ab-0eb3-4477-81e6-af5fdc86d18a-maas-api-service-ca\") pod \"c387d6ab-0eb3-4477-81e6-af5fdc86d18a\" (UID: \"c387d6ab-0eb3-4477-81e6-af5fdc86d18a\") "
Sep 16 16:54:02.186576 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:02.186552    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c387d6ab-0eb3-4477-81e6-af5fdc86d18a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c387d6ab-0eb3-4477-81e6-af5fdc86d18a" (UID: "c387d6ab-0eb3-4477-81e6-af5fdc86d18a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:02.286922 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:02.286884    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c387d6ab-0eb3-4477-81e6-af5fdc86d18a-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:02.317115 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:02.317076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:02.943687 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:02.943656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844" event={"ID":"c387d6ab-0eb3-4477-81e6-af5fdc86d18a","Type":"ContainerDied","Data":"d50a348a5db398d451b707c1a6cfdc704d02ae030564408a3fa31d721071e6ca"}
Sep 16 16:54:02.943687 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:02.943687    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"
Sep 16 16:54:02.943879 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:02.943703    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d50a348a5db398d451b707c1a6cfdc704d02ae030564408a3fa31d721071e6ca"
Sep 16 16:54:03.192851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:03.192819    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:03.907002 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:03.906967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:04.753264 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:04.753225    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:06.776072 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:06.776021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:07.419812 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:07.419776    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:07.474859 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:07.474828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:10.337835 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.337797    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:10.362980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.362950    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"]
Sep 16 16:54:10.363285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.363252    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s" podUID="f5655cc5-a18e-4636-9523-e6293c18627c" containerName="payload-processing" containerID="cri-o://dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1" gracePeriod=30
Sep 16 16:54:10.373614 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.373517    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"]
Sep 16 16:54:10.373811 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.373775    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l" podUID="5ed33ca0-b2ea-4ca3-8813-655c6152f56d" containerName="payload-pre-processing" containerID="cri-o://3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6" gracePeriod=30
Sep 16 16:54:10.438325 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.438294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:10.540218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.539369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:10.554412 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.554241    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"]
Sep 16 16:54:10.557651 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.557627    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1fd647e0-k7844"]
Sep 16 16:54:10.638387 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.638349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:10.653548 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.653516    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:54:10.656503 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.656484    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:54:10.766808 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.766775    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs66q\" (UniqueName: \"kubernetes.io/projected/f5655cc5-a18e-4636-9523-e6293c18627c-kube-api-access-bs66q\") pod \"f5655cc5-a18e-4636-9523-e6293c18627c\" (UID: \"f5655cc5-a18e-4636-9523-e6293c18627c\") "
Sep 16 16:54:10.766808 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.766812    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82ppz\" (UniqueName: \"kubernetes.io/projected/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-kube-api-access-82ppz\") pod \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\" (UID: \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\") "
Sep 16 16:54:10.767054 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.766854    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-plugins-config-volume\") pod \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\" (UID: \"5ed33ca0-b2ea-4ca3-8813-655c6152f56d\") "
Sep 16 16:54:10.767054 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.766875    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5655cc5-a18e-4636-9523-e6293c18627c-plugins-config-volume\") pod \"f5655cc5-a18e-4636-9523-e6293c18627c\" (UID: \"f5655cc5-a18e-4636-9523-e6293c18627c\") "
Sep 16 16:54:10.767258 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.767231    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ed33ca0-b2ea-4ca3-8813-655c6152f56d" (UID: "5ed33ca0-b2ea-4ca3-8813-655c6152f56d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:10.767324 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.767279    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5655cc5-a18e-4636-9523-e6293c18627c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f5655cc5-a18e-4636-9523-e6293c18627c" (UID: "f5655cc5-a18e-4636-9523-e6293c18627c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:10.768906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.768886    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5655cc5-a18e-4636-9523-e6293c18627c-kube-api-access-bs66q" (OuterVolumeSpecName: "kube-api-access-bs66q") pod "f5655cc5-a18e-4636-9523-e6293c18627c" (UID: "f5655cc5-a18e-4636-9523-e6293c18627c"). InnerVolumeSpecName "kube-api-access-bs66q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:10.768991 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.768911    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-kube-api-access-82ppz" (OuterVolumeSpecName: "kube-api-access-82ppz") pod "5ed33ca0-b2ea-4ca3-8813-655c6152f56d" (UID: "5ed33ca0-b2ea-4ca3-8813-655c6152f56d"). InnerVolumeSpecName "kube-api-access-82ppz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:10.867589 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.867557    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs66q\" (UniqueName: \"kubernetes.io/projected/f5655cc5-a18e-4636-9523-e6293c18627c-kube-api-access-bs66q\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:10.867589 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.867586    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82ppz\" (UniqueName: \"kubernetes.io/projected/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-kube-api-access-82ppz\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:10.867589 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.867596    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ed33ca0-b2ea-4ca3-8813-655c6152f56d-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:10.867806 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.867605    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5655cc5-a18e-4636-9523-e6293c18627c-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:10.979128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:10.979094    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c387d6ab-0eb3-4477-81e6-af5fdc86d18a" path="/var/lib/kubelet/pods/c387d6ab-0eb3-4477-81e6-af5fdc86d18a/volumes"
Sep 16 16:54:11.000655 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.000625    2577 generic.go:356] "Generic (PLEG): container finished" podID="f5655cc5-a18e-4636-9523-e6293c18627c" containerID="dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1" exitCode=0
Sep 16 16:54:11.000792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.000696    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"
Sep 16 16:54:11.000792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.000715    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s" event={"ID":"f5655cc5-a18e-4636-9523-e6293c18627c","Type":"ContainerDied","Data":"dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1"}
Sep 16 16:54:11.000792 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.000778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s" event={"ID":"f5655cc5-a18e-4636-9523-e6293c18627c","Type":"ContainerDied","Data":"be54770a2b0dcbe1edb211da40d8bb0080b8273995ebbed868da9d719f5abfca"}
Sep 16 16:54:11.000986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.000805    2577 scope.go:117] "RemoveContainer" containerID="dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1"
Sep 16 16:54:11.002343 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.002321    2577 generic.go:356] "Generic (PLEG): container finished" podID="5ed33ca0-b2ea-4ca3-8813-655c6152f56d" containerID="3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6" exitCode=0
Sep 16 16:54:11.002459 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.002373    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"
Sep 16 16:54:11.002459 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.002415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l" event={"ID":"5ed33ca0-b2ea-4ca3-8813-655c6152f56d","Type":"ContainerDied","Data":"3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6"}
Sep 16 16:54:11.002606 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.002462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l" event={"ID":"5ed33ca0-b2ea-4ca3-8813-655c6152f56d","Type":"ContainerDied","Data":"e96bb8cf9a5b6039d594a7893d2253b645d28c847e62131e2c65fb2d0371b417"}
Sep 16 16:54:11.010759 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.010733    2577 scope.go:117] "RemoveContainer" containerID="dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1"
Sep 16 16:54:11.011033 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:54:11.011016    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1\": container with ID starting with dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1 not found: ID does not exist" containerID="dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1"
Sep 16 16:54:11.011078 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.011041    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1"} err="failed to get container status \"dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1\": rpc error: code = NotFound desc = could not find container \"dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1\": container with ID starting with dc4259fc118e34fbc2e7e2a9b21caba99c4515ade5d8304e1d8b8431050ad4b1 not found: ID does not exist"
Sep 16 16:54:11.011078 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.011056    2577 scope.go:117] "RemoveContainer" containerID="3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6"
Sep 16 16:54:11.019900 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.019884    2577 scope.go:117] "RemoveContainer" containerID="3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6"
Sep 16 16:54:11.020169 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:54:11.020148    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6\": container with ID starting with 3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6 not found: ID does not exist" containerID="3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6"
Sep 16 16:54:11.020211 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.020187    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6"} err="failed to get container status \"3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6\": rpc error: code = NotFound desc = could not find container \"3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6\": container with ID starting with 3e782fe22d59586b08a03e977ec63eff547162a097d6e35f967a6890803597a6 not found: ID does not exist"
Sep 16 16:54:11.030593 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.030569    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"]
Sep 16 16:54:11.034845 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.034815    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1fd647e0-65dc4cfbd5-bhx5s"]
Sep 16 16:54:11.053296 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.053274    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"]
Sep 16 16:54:11.057727 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.057706    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1fd647e0-8497bbc794-m7z5l"]
Sep 16 16:54:11.058454 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.058430    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:11.674928 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.674886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:11.913516 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:11.913479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:12.118909 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:12.118822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:12.440441 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:12.440410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:12.726265 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:12.726169    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:12.978269 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:12.978187    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ed33ca0-b2ea-4ca3-8813-655c6152f56d" path="/var/lib/kubelet/pods/5ed33ca0-b2ea-4ca3-8813-655c6152f56d/volumes"
Sep 16 16:54:12.978618 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:12.978602    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5655cc5-a18e-4636-9523-e6293c18627c" path="/var/lib/kubelet/pods/f5655cc5-a18e-4636-9523-e6293c18627c/volumes"
Sep 16 16:54:13.023747 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:13.023715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:13.769083 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:13.769036    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:14.533055 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:14.533022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:15.067492 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:15.067458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:15.236581 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:15.236520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:15.801442 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:15.801396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:17.025620 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:17.025578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:17.154438 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:17.154404    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:17.802337 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:17.802300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:18.014468 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.014431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:18.161050 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.161021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:18.220208 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220171    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"]
Sep 16 16:54:18.220697 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220678    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5655cc5-a18e-4636-9523-e6293c18627c" containerName="payload-processing"
Sep 16 16:54:18.220697 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220698    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5655cc5-a18e-4636-9523-e6293c18627c" containerName="payload-processing"
Sep 16 16:54:18.220870 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220718    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed33ca0-b2ea-4ca3-8813-655c6152f56d" containerName="payload-pre-processing"
Sep 16 16:54:18.220870 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220727    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed33ca0-b2ea-4ca3-8813-655c6152f56d" containerName="payload-pre-processing"
Sep 16 16:54:18.220870 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220773    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c387d6ab-0eb3-4477-81e6-af5fdc86d18a" containerName="revoke-keys"
Sep 16 16:54:18.220870 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220784    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c387d6ab-0eb3-4477-81e6-af5fdc86d18a" containerName="revoke-keys"
Sep 16 16:54:18.221060 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220879    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c387d6ab-0eb3-4477-81e6-af5fdc86d18a" containerName="revoke-keys"
Sep 16 16:54:18.221060 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220893    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed33ca0-b2ea-4ca3-8813-655c6152f56d" containerName="payload-pre-processing"
Sep 16 16:54:18.221060 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.220911    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5655cc5-a18e-4636-9523-e6293c18627c" containerName="payload-processing"
Sep 16 16:54:18.225738 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.225719    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:18.229368 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.229349    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-b2d51e27\""
Sep 16 16:54:18.229470 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.229411    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-b2d51e27-dockercfg-kk8pk\""
Sep 16 16:54:18.239360 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.239338    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"]
Sep 16 16:54:18.267101 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.267067    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"]
Sep 16 16:54:18.270510 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.270490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:18.284163 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.284141    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"]
Sep 16 16:54:18.335280 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.335253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbv7k\" (UniqueName: \"kubernetes.io/projected/0ae86643-440b-4baf-b302-6a7432950b9e-kube-api-access-vbv7k\") pod \"payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g\" (UID: \"0ae86643-440b-4baf-b302-6a7432950b9e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:18.335434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.335306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae86643-440b-4baf-b302-6a7432950b9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g\" (UID: \"0ae86643-440b-4baf-b302-6a7432950b9e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:18.335434 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.335330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a53b83-0041-4011-a5cd-03eaf6b38ece-plugins-config-volume\") pod \"payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb\" (UID: \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\") " pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:18.335574 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.335435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drzjr\" (UniqueName: \"kubernetes.io/projected/e3a53b83-0041-4011-a5cd-03eaf6b38ece-kube-api-access-drzjr\") pod \"payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb\" (UID: \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\") " pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:18.436128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.436063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbv7k\" (UniqueName: \"kubernetes.io/projected/0ae86643-440b-4baf-b302-6a7432950b9e-kube-api-access-vbv7k\") pod \"payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g\" (UID: \"0ae86643-440b-4baf-b302-6a7432950b9e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:18.436128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.436113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae86643-440b-4baf-b302-6a7432950b9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g\" (UID: \"0ae86643-440b-4baf-b302-6a7432950b9e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:18.436288 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.436135    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a53b83-0041-4011-a5cd-03eaf6b38ece-plugins-config-volume\") pod \"payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb\" (UID: \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\") " pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:18.436288 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.436184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drzjr\" (UniqueName: \"kubernetes.io/projected/e3a53b83-0041-4011-a5cd-03eaf6b38ece-kube-api-access-drzjr\") pod \"payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb\" (UID: \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\") " pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:18.437007 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.436982    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae86643-440b-4baf-b302-6a7432950b9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g\" (UID: \"0ae86643-440b-4baf-b302-6a7432950b9e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:18.437051 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.436985    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a53b83-0041-4011-a5cd-03eaf6b38ece-plugins-config-volume\") pod \"payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb\" (UID: \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\") " pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:18.454247 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.454222    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbv7k\" (UniqueName: \"kubernetes.io/projected/0ae86643-440b-4baf-b302-6a7432950b9e-kube-api-access-vbv7k\") pod \"payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g\" (UID: \"0ae86643-440b-4baf-b302-6a7432950b9e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:18.454773 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.454751    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drzjr\" (UniqueName: \"kubernetes.io/projected/e3a53b83-0041-4011-a5cd-03eaf6b38ece-kube-api-access-drzjr\") pod \"payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb\" (UID: \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\") " pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:18.536400 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.536374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:18.580326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.580294    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:18.653478 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.653245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:18.817593 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.817560    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"]
Sep 16 16:54:18.818098 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:54:18.818076    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae86643_440b_4baf_b302_6a7432950b9e.slice/crio-9779f7f2150b290a44cf9160bdff2189111b42cd041cc6f64dbde1cdb57485ff WatchSource:0}: Error finding container 9779f7f2150b290a44cf9160bdff2189111b42cd041cc6f64dbde1cdb57485ff: Status 404 returned error can't find the container with id 9779f7f2150b290a44cf9160bdff2189111b42cd041cc6f64dbde1cdb57485ff
Sep 16 16:54:18.878169 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:18.878141    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"]
Sep 16 16:54:18.879014 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:54:18.878990    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a53b83_0041_4011_a5cd_03eaf6b38ece.slice/crio-5ca3a73dbfc8e8df132f0b36fbf8d63505689642884730d6558434912c8df0e4 WatchSource:0}: Error finding container 5ca3a73dbfc8e8df132f0b36fbf8d63505689642884730d6558434912c8df0e4: Status 404 returned error can't find the container with id 5ca3a73dbfc8e8df132f0b36fbf8d63505689642884730d6558434912c8df0e4
Sep 16 16:54:19.055342 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:19.055303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb" event={"ID":"e3a53b83-0041-4011-a5cd-03eaf6b38ece","Type":"ContainerStarted","Data":"b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784"}
Sep 16 16:54:19.055545 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:19.055365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb" event={"ID":"e3a53b83-0041-4011-a5cd-03eaf6b38ece","Type":"ContainerStarted","Data":"5ca3a73dbfc8e8df132f0b36fbf8d63505689642884730d6558434912c8df0e4"}
Sep 16 16:54:19.055617 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:19.055545    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:19.057053 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:19.057029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g" event={"ID":"0ae86643-440b-4baf-b302-6a7432950b9e","Type":"ContainerStarted","Data":"ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951"}
Sep 16 16:54:19.057157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:19.057066    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g" event={"ID":"0ae86643-440b-4baf-b302-6a7432950b9e","Type":"ContainerStarted","Data":"9779f7f2150b290a44cf9160bdff2189111b42cd041cc6f64dbde1cdb57485ff"}
Sep 16 16:54:19.057157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:19.057085    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:19.088254 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:19.088172    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb" podStartSLOduration=1.088154946 podStartE2EDuration="1.088154946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:54:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:54:19.087302456 +0000 UTC m=+1074.584175579" watchObservedRunningTime="2026-09-16 16:54:19.088154946 +0000 UTC m=+1074.585028072"
Sep 16 16:54:19.872431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:19.872373    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:20.894502 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:20.894469    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:21.890905 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:21.890873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:22.652449 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:22.652418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:30.065743 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:30.065710    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:30.066192 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:30.065803    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:30.101392 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:30.101341    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g" podStartSLOduration=12.101326891 podStartE2EDuration="12.101326891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:54:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:54:19.120126059 +0000 UTC m=+1074.616999184" watchObservedRunningTime="2026-09-16 16:54:30.101326891 +0000 UTC m=+1085.598200015"
Sep 16 16:54:34.906115 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:34.906080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:39.667597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.667562    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"]
Sep 16 16:54:39.673193 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.673167    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"
Sep 16 16:54:39.676130 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.676104    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:54:39.677180 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.677106    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"]
Sep 16 16:54:39.677303 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.677258    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:54:39.733201 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.733167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9a3cad-1846-484c-98e7-830b861dd1b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn\" (UID: \"6b9a3cad-1846-484c-98e7-830b861dd1b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"
Sep 16 16:54:39.834538 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.834493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9a3cad-1846-484c-98e7-830b861dd1b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn\" (UID: \"6b9a3cad-1846-484c-98e7-830b861dd1b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"
Sep 16 16:54:39.835094 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.835075    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9a3cad-1846-484c-98e7-830b861dd1b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn\" (UID: \"6b9a3cad-1846-484c-98e7-830b861dd1b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"
Sep 16 16:54:39.986926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:39.986851    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"
Sep 16 16:54:40.198647 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:40.198616    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:40.281743 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:54:40.281703    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b9a3cad_1846_484c_98e7_830b861dd1b4.slice/crio-e8ee3748a61854340bf58983e5a2cc55aea75df5ebee7315aad31a7f8fdb7e47 WatchSource:0}: Error finding container e8ee3748a61854340bf58983e5a2cc55aea75df5ebee7315aad31a7f8fdb7e47: Status 404 returned error can't find the container with id e8ee3748a61854340bf58983e5a2cc55aea75df5ebee7315aad31a7f8fdb7e47
Sep 16 16:54:40.283022 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:40.282997    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"]
Sep 16 16:54:41.028538 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:41.028506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:41.194522 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:41.194489    2577 generic.go:356] "Generic (PLEG): container finished" podID="6b9a3cad-1846-484c-98e7-830b861dd1b4" containerID="ed3bbac4ab347c7b305eecf1372b72853841bfc870b4237d3dc2c99ee0f0cba0" exitCode=0
Sep 16 16:54:41.194741 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:41.194582    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn" event={"ID":"6b9a3cad-1846-484c-98e7-830b861dd1b4","Type":"ContainerDied","Data":"ed3bbac4ab347c7b305eecf1372b72853841bfc870b4237d3dc2c99ee0f0cba0"}
Sep 16 16:54:41.194741 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:41.194647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn" event={"ID":"6b9a3cad-1846-484c-98e7-830b861dd1b4","Type":"ContainerStarted","Data":"e8ee3748a61854340bf58983e5a2cc55aea75df5ebee7315aad31a7f8fdb7e47"}
Sep 16 16:54:41.760666 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:41.760627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:42.114643 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:42.114552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:42.348703 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:42.348674    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"
Sep 16 16:54:42.458811 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:42.458783    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9a3cad-1846-484c-98e7-830b861dd1b4-maas-api-service-ca\") pod \"6b9a3cad-1846-484c-98e7-830b861dd1b4\" (UID: \"6b9a3cad-1846-484c-98e7-830b861dd1b4\") "
Sep 16 16:54:42.459126 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:42.459106    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b9a3cad-1846-484c-98e7-830b861dd1b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6b9a3cad-1846-484c-98e7-830b861dd1b4" (UID: "6b9a3cad-1846-484c-98e7-830b861dd1b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:42.559687 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:42.559647    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b9a3cad-1846-484c-98e7-830b861dd1b4-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:42.684408 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:42.684376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:43.208165 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:43.208130    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"
Sep 16 16:54:43.208165 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:43.208146    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn" event={"ID":"6b9a3cad-1846-484c-98e7-830b861dd1b4","Type":"ContainerDied","Data":"e8ee3748a61854340bf58983e5a2cc55aea75df5ebee7315aad31a7f8fdb7e47"}
Sep 16 16:54:43.208614 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:43.208208    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8ee3748a61854340bf58983e5a2cc55aea75df5ebee7315aad31a7f8fdb7e47"
Sep 16 16:54:43.489524 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:43.489474    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:44.309921 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:44.309886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:45.015839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:45.015803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:46.734579 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.734542    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:46.771714 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.771668    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"]
Sep 16 16:54:46.772011 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.771981    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb" podUID="e3a53b83-0041-4011-a5cd-03eaf6b38ece" containerName="payload-processing" containerID="cri-o://b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784" gracePeriod=30
Sep 16 16:54:46.775377 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.775110    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"]
Sep 16 16:54:46.775377 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.775287    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g" podUID="0ae86643-440b-4baf-b302-6a7432950b9e" containerName="payload-pre-processing" containerID="cri-o://ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951" gracePeriod=30
Sep 16 16:54:46.818826 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.818795    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:46.900862 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.900816    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:46.976517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.976456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:46.980489 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.980456    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"]
Sep 16 16:54:46.980658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:46.980499    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b2d51e27-l8psn"]
Sep 16 16:54:47.028285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.028262    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:47.050856 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.050759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:47.059521 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.059494    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:47.100072 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.100044    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbv7k\" (UniqueName: \"kubernetes.io/projected/0ae86643-440b-4baf-b302-6a7432950b9e-kube-api-access-vbv7k\") pod \"0ae86643-440b-4baf-b302-6a7432950b9e\" (UID: \"0ae86643-440b-4baf-b302-6a7432950b9e\") "
Sep 16 16:54:47.100243 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.100095    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drzjr\" (UniqueName: \"kubernetes.io/projected/e3a53b83-0041-4011-a5cd-03eaf6b38ece-kube-api-access-drzjr\") pod \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\" (UID: \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\") "
Sep 16 16:54:47.100243 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.100212    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae86643-440b-4baf-b302-6a7432950b9e-plugins-config-volume\") pod \"0ae86643-440b-4baf-b302-6a7432950b9e\" (UID: \"0ae86643-440b-4baf-b302-6a7432950b9e\") "
Sep 16 16:54:47.100373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.100331    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a53b83-0041-4011-a5cd-03eaf6b38ece-plugins-config-volume\") pod \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\" (UID: \"e3a53b83-0041-4011-a5cd-03eaf6b38ece\") "
Sep 16 16:54:47.100709 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.100681    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae86643-440b-4baf-b302-6a7432950b9e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ae86643-440b-4baf-b302-6a7432950b9e" (UID: "0ae86643-440b-4baf-b302-6a7432950b9e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:47.100820 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.100784    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3a53b83-0041-4011-a5cd-03eaf6b38ece-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3a53b83-0041-4011-a5cd-03eaf6b38ece" (UID: "e3a53b83-0041-4011-a5cd-03eaf6b38ece"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:54:47.103130 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.103105    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae86643-440b-4baf-b302-6a7432950b9e-kube-api-access-vbv7k" (OuterVolumeSpecName: "kube-api-access-vbv7k") pod "0ae86643-440b-4baf-b302-6a7432950b9e" (UID: "0ae86643-440b-4baf-b302-6a7432950b9e"). InnerVolumeSpecName "kube-api-access-vbv7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:47.104601 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.104569    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3a53b83-0041-4011-a5cd-03eaf6b38ece-kube-api-access-drzjr" (OuterVolumeSpecName: "kube-api-access-drzjr") pod "e3a53b83-0041-4011-a5cd-03eaf6b38ece" (UID: "e3a53b83-0041-4011-a5cd-03eaf6b38ece"). InnerVolumeSpecName "kube-api-access-drzjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:54:47.142008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.141975    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:47.201899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.201867    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae86643-440b-4baf-b302-6a7432950b9e-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:47.201899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.201893    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a53b83-0041-4011-a5cd-03eaf6b38ece-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:47.201899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.201903    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbv7k\" (UniqueName: \"kubernetes.io/projected/0ae86643-440b-4baf-b302-6a7432950b9e-kube-api-access-vbv7k\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:47.202167 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.201911    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drzjr\" (UniqueName: \"kubernetes.io/projected/e3a53b83-0041-4011-a5cd-03eaf6b38ece-kube-api-access-drzjr\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:54:47.232464 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.232437    2577 generic.go:356] "Generic (PLEG): container finished" podID="e3a53b83-0041-4011-a5cd-03eaf6b38ece" containerID="b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784" exitCode=0
Sep 16 16:54:47.232626 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.232473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb" event={"ID":"e3a53b83-0041-4011-a5cd-03eaf6b38ece","Type":"ContainerDied","Data":"b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784"}
Sep 16 16:54:47.232626 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.232499    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"
Sep 16 16:54:47.232626 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.232555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb" event={"ID":"e3a53b83-0041-4011-a5cd-03eaf6b38ece","Type":"ContainerDied","Data":"5ca3a73dbfc8e8df132f0b36fbf8d63505689642884730d6558434912c8df0e4"}
Sep 16 16:54:47.232626 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.232593    2577 scope.go:117] "RemoveContainer" containerID="b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784"
Sep 16 16:54:47.233991 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.233972    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ae86643-440b-4baf-b302-6a7432950b9e" containerID="ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951" exitCode=0
Sep 16 16:54:47.234079 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.234022    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"
Sep 16 16:54:47.234079 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.234048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g" event={"ID":"0ae86643-440b-4baf-b302-6a7432950b9e","Type":"ContainerDied","Data":"ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951"}
Sep 16 16:54:47.234183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.234092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g" event={"ID":"0ae86643-440b-4baf-b302-6a7432950b9e","Type":"ContainerDied","Data":"9779f7f2150b290a44cf9160bdff2189111b42cd041cc6f64dbde1cdb57485ff"}
Sep 16 16:54:47.245222 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.245201    2577 scope.go:117] "RemoveContainer" containerID="b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784"
Sep 16 16:54:47.245512 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:54:47.245472    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784\": container with ID starting with b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784 not found: ID does not exist" containerID="b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784"
Sep 16 16:54:47.245731 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.245510    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784"} err="failed to get container status \"b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784\": rpc error: code = NotFound desc = could not find container \"b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784\": container with ID starting with b1dfa85deb7dc7e276f4c254645a4af792be9fa21c3c1c06f0abe88c393e7784 not found: ID does not exist"
Sep 16 16:54:47.245731 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.245549    2577 scope.go:117] "RemoveContainer" containerID="ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951"
Sep 16 16:54:47.254302 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.254283    2577 scope.go:117] "RemoveContainer" containerID="ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951"
Sep 16 16:54:47.254656 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:54:47.254561    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951\": container with ID starting with ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951 not found: ID does not exist" containerID="ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951"
Sep 16 16:54:47.254656 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.254583    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951"} err="failed to get container status \"ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951\": rpc error: code = NotFound desc = could not find container \"ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951\": container with ID starting with ab20fd449eca7a3285b0effa5b9c61c2bab2ef0d8f6577388476296adba96951 not found: ID does not exist"
Sep 16 16:54:47.261691 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.261671    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"]
Sep 16 16:54:47.265369 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.265348    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-b2d51e27-7745b7ff8-s482g"]
Sep 16 16:54:47.276349 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.276329    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"]
Sep 16 16:54:47.282083 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.282061    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b2d51e27-55d5c48b97-ffrdb"]
Sep 16 16:54:47.480177 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.480144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:47.767410 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.767374    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:47.865442 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.865401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:47.965146 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:47.965109    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:48.455765 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:48.455727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:48.979041 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:48.979004    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ae86643-440b-4baf-b302-6a7432950b9e" path="/var/lib/kubelet/pods/0ae86643-440b-4baf-b302-6a7432950b9e/volumes"
Sep 16 16:54:48.979470 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:48.979382    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b9a3cad-1846-484c-98e7-830b861dd1b4" path="/var/lib/kubelet/pods/6b9a3cad-1846-484c-98e7-830b861dd1b4/volumes"
Sep 16 16:54:48.979737 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:48.979723    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3a53b83-0041-4011-a5cd-03eaf6b38ece" path="/var/lib/kubelet/pods/e3a53b83-0041-4011-a5cd-03eaf6b38ece/volumes"
Sep 16 16:54:49.280618 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:49.280510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:49.405164 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:49.405124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:50.148487 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:50.148444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:50.976713 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:50.976676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:51.679523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:51.679478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:55.505281 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:55.505240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:55.608084 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:55.608040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:55.718715 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:55.718682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:55.825633 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:55.825552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:54:56.271489 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:54:56.271448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:02.773906 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:02.773809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:03.105888 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:03.105791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:03.651922 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:03.651871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:05.606732 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:05.606701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:13.433408 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:13.433371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:14.278048 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:14.278015    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:15.082480 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:15.082432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:16.153235 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:16.153182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:16.679523 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:16.679484    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:17.672572 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:17.672510    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:18.400507 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:18.400472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:22.161822 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:22.161789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:24.042020 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.041981    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:24.068734 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.068700    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"]
Sep 16 16:55:24.068989 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.068960    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt" podUID="ddec1d46-4623-4d8d-a17d-3747b4ddef16" containerName="payload-processing" containerID="cri-o://b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363" gracePeriod=30
Sep 16 16:55:24.077928 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.076051    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"]
Sep 16 16:55:24.077928 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.076238    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2" podUID="e2265a70-f468-4d97-b29a-ba4423fabe6f" containerName="payload-pre-processing" containerID="cri-o://49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291" gracePeriod=30
Sep 16 16:55:24.111148 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.111122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:24.180347 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.180251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:24.319423 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.319386    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:24.327106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.327083    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:55:24.357428 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.357402    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddec1d46-4623-4d8d-a17d-3747b4ddef16-plugins-config-volume\") pod \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\" (UID: \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\") "
Sep 16 16:55:24.357594 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.357473    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ls7vp\" (UniqueName: \"kubernetes.io/projected/ddec1d46-4623-4d8d-a17d-3747b4ddef16-kube-api-access-ls7vp\") pod \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\" (UID: \"ddec1d46-4623-4d8d-a17d-3747b4ddef16\") "
Sep 16 16:55:24.357922 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.357896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddec1d46-4623-4d8d-a17d-3747b4ddef16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ddec1d46-4623-4d8d-a17d-3747b4ddef16" (UID: "ddec1d46-4623-4d8d-a17d-3747b4ddef16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:24.359867 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.359840    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddec1d46-4623-4d8d-a17d-3747b4ddef16-kube-api-access-ls7vp" (OuterVolumeSpecName: "kube-api-access-ls7vp") pod "ddec1d46-4623-4d8d-a17d-3747b4ddef16" (UID: "ddec1d46-4623-4d8d-a17d-3747b4ddef16"). InnerVolumeSpecName "kube-api-access-ls7vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:24.361786 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.361766    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:55:24.389706 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.389672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:24.457988 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.457956    2577 generic.go:356] "Generic (PLEG): container finished" podID="e2265a70-f468-4d97-b29a-ba4423fabe6f" containerID="49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291" exitCode=0
Sep 16 16:55:24.458128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.457995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2" event={"ID":"e2265a70-f468-4d97-b29a-ba4423fabe6f","Type":"ContainerDied","Data":"49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291"}
Sep 16 16:55:24.458128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.458018    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"
Sep 16 16:55:24.458128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.458053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2" event={"ID":"e2265a70-f468-4d97-b29a-ba4423fabe6f","Type":"ContainerDied","Data":"d9078e95c7b4fff149f212c87c75127b1c09c4a01b71330626e6f61fa584365a"}
Sep 16 16:55:24.458128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.458082    2577 scope.go:117] "RemoveContainer" containerID="49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291"
Sep 16 16:55:24.458128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.458089    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjc74\" (UniqueName: \"kubernetes.io/projected/e2265a70-f468-4d97-b29a-ba4423fabe6f-kube-api-access-gjc74\") pod \"e2265a70-f468-4d97-b29a-ba4423fabe6f\" (UID: \"e2265a70-f468-4d97-b29a-ba4423fabe6f\") "
Sep 16 16:55:24.458128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.458123    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2265a70-f468-4d97-b29a-ba4423fabe6f-plugins-config-volume\") pod \"e2265a70-f468-4d97-b29a-ba4423fabe6f\" (UID: \"e2265a70-f468-4d97-b29a-ba4423fabe6f\") "
Sep 16 16:55:24.458496 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.458473    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddec1d46-4623-4d8d-a17d-3747b4ddef16-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:24.458592 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.458503    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls7vp\" (UniqueName: \"kubernetes.io/projected/ddec1d46-4623-4d8d-a17d-3747b4ddef16-kube-api-access-ls7vp\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:24.458692 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.458658    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2265a70-f468-4d97-b29a-ba4423fabe6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2265a70-f468-4d97-b29a-ba4423fabe6f" (UID: "e2265a70-f468-4d97-b29a-ba4423fabe6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:24.459666 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.459637    2577 generic.go:356] "Generic (PLEG): container finished" podID="ddec1d46-4623-4d8d-a17d-3747b4ddef16" containerID="b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363" exitCode=0
Sep 16 16:55:24.459785 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.459683    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"
Sep 16 16:55:24.459785 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.459717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt" event={"ID":"ddec1d46-4623-4d8d-a17d-3747b4ddef16","Type":"ContainerDied","Data":"b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363"}
Sep 16 16:55:24.459785 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.459768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt" event={"ID":"ddec1d46-4623-4d8d-a17d-3747b4ddef16","Type":"ContainerDied","Data":"17f0505d495d3dee4d6e5637c285f1ac41b5984c1b4a7ef61b15c9cd142ef962"}
Sep 16 16:55:24.460853 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.460824    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2265a70-f468-4d97-b29a-ba4423fabe6f-kube-api-access-gjc74" (OuterVolumeSpecName: "kube-api-access-gjc74") pod "e2265a70-f468-4d97-b29a-ba4423fabe6f" (UID: "e2265a70-f468-4d97-b29a-ba4423fabe6f"). InnerVolumeSpecName "kube-api-access-gjc74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:24.468014 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.467907    2577 scope.go:117] "RemoveContainer" containerID="49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291"
Sep 16 16:55:24.468349 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:55:24.468316    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291\": container with ID starting with 49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291 not found: ID does not exist" containerID="49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291"
Sep 16 16:55:24.468449 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.468357    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291"} err="failed to get container status \"49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291\": rpc error: code = NotFound desc = could not find container \"49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291\": container with ID starting with 49f2d6c09f5af123bf8883f1a48be46131ffc5994e0eba4e7afe5273b0ac4291 not found: ID does not exist"
Sep 16 16:55:24.468449 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.468375    2577 scope.go:117] "RemoveContainer" containerID="b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363"
Sep 16 16:55:24.476571 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.476554    2577 scope.go:117] "RemoveContainer" containerID="b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363"
Sep 16 16:55:24.476830 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:55:24.476811    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363\": container with ID starting with b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363 not found: ID does not exist" containerID="b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363"
Sep 16 16:55:24.476896 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.476835    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363"} err="failed to get container status \"b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363\": rpc error: code = NotFound desc = could not find container \"b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363\": container with ID starting with b9e963028dd308ab7256735aceec35c5861de2351ca7f154db0f441d5f6a9363 not found: ID does not exist"
Sep 16 16:55:24.483067 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.483044    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"]
Sep 16 16:55:24.487987 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.487967    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-708c6a-68c9db4cd7-5nnvt"]
Sep 16 16:55:24.559268 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.559190    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjc74\" (UniqueName: \"kubernetes.io/projected/e2265a70-f468-4d97-b29a-ba4423fabe6f-kube-api-access-gjc74\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:24.559268 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.559221    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2265a70-f468-4d97-b29a-ba4423fabe6f-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:24.579660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.579629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:24.686105 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.686041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:24.781439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.781408    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"]
Sep 16 16:55:24.784669 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.784645    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-708c6a-7df7f7d7b6-mp5j2"]
Sep 16 16:55:24.978121 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.978085    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddec1d46-4623-4d8d-a17d-3747b4ddef16" path="/var/lib/kubelet/pods/ddec1d46-4623-4d8d-a17d-3747b4ddef16/volumes"
Sep 16 16:55:24.978557 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:24.978521    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2265a70-f468-4d97-b29a-ba4423fabe6f" path="/var/lib/kubelet/pods/e2265a70-f468-4d97-b29a-ba4423fabe6f/volumes"
Sep 16 16:55:25.410472 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:25.410436    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:26.136162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:26.136126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:26.789250 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:26.789218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:27.020583 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.020545    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"]
Sep 16 16:55:27.020964 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.020952    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2265a70-f468-4d97-b29a-ba4423fabe6f" containerName="payload-pre-processing"
Sep 16 16:55:27.021008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.020965    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2265a70-f468-4d97-b29a-ba4423fabe6f" containerName="payload-pre-processing"
Sep 16 16:55:27.021008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.020980    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddec1d46-4623-4d8d-a17d-3747b4ddef16" containerName="payload-processing"
Sep 16 16:55:27.021008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.020986    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddec1d46-4623-4d8d-a17d-3747b4ddef16" containerName="payload-processing"
Sep 16 16:55:27.021008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.020997    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3a53b83-0041-4011-a5cd-03eaf6b38ece" containerName="payload-processing"
Sep 16 16:55:27.021008 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021003    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3a53b83-0041-4011-a5cd-03eaf6b38ece" containerName="payload-processing"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021016    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b9a3cad-1846-484c-98e7-830b861dd1b4" containerName="revoke-keys"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021021    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b9a3cad-1846-484c-98e7-830b861dd1b4" containerName="revoke-keys"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021032    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae86643-440b-4baf-b302-6a7432950b9e" containerName="payload-pre-processing"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021037    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae86643-440b-4baf-b302-6a7432950b9e" containerName="payload-pre-processing"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021111    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae86643-440b-4baf-b302-6a7432950b9e" containerName="payload-pre-processing"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021120    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddec1d46-4623-4d8d-a17d-3747b4ddef16" containerName="payload-processing"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021126    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3a53b83-0041-4011-a5cd-03eaf6b38ece" containerName="payload-processing"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021134    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2265a70-f468-4d97-b29a-ba4423fabe6f" containerName="payload-pre-processing"
Sep 16 16:55:27.021162 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.021138    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b9a3cad-1846-484c-98e7-830b861dd1b4" containerName="revoke-keys"
Sep 16 16:55:27.025579 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.025562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"
Sep 16 16:55:27.028512 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.028488    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:55:27.028680 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.028488    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:55:27.031123 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.031098    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"]
Sep 16 16:55:27.083766 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.083693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c205c3a7-9034-44d0-a18c-dab2a9aa4ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc\" (UID: \"c205c3a7-9034-44d0-a18c-dab2a9aa4ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"
Sep 16 16:55:27.184476 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.184438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c205c3a7-9034-44d0-a18c-dab2a9aa4ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc\" (UID: \"c205c3a7-9034-44d0-a18c-dab2a9aa4ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"
Sep 16 16:55:27.185096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.185076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c205c3a7-9034-44d0-a18c-dab2a9aa4ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc\" (UID: \"c205c3a7-9034-44d0-a18c-dab2a9aa4ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"
Sep 16 16:55:27.231025 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.230983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:27.338876 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.338794    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"
Sep 16 16:55:27.615335 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:27.615303    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"]
Sep 16 16:55:27.616434 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:55:27.616402    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc205c3a7_9034_44d0_a18c_dab2a9aa4ab0.slice/crio-10a4401fa6a8ab247f81b94247300a80f1dfc87e4fbfc83aff198acfb188b46d WatchSource:0}: Error finding container 10a4401fa6a8ab247f81b94247300a80f1dfc87e4fbfc83aff198acfb188b46d: Status 404 returned error can't find the container with id 10a4401fa6a8ab247f81b94247300a80f1dfc87e4fbfc83aff198acfb188b46d
Sep 16 16:55:28.079726 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:28.079692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:28.488326 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:28.488288    2577 generic.go:356] "Generic (PLEG): container finished" podID="c205c3a7-9034-44d0-a18c-dab2a9aa4ab0" containerID="7e062ad61f3e49fc752f3906722143a4b819755520f2f29f8749d8d988e0a46f" exitCode=0
Sep 16 16:55:28.488520 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:28.488345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc" event={"ID":"c205c3a7-9034-44d0-a18c-dab2a9aa4ab0","Type":"ContainerDied","Data":"7e062ad61f3e49fc752f3906722143a4b819755520f2f29f8749d8d988e0a46f"}
Sep 16 16:55:28.488520 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:28.488400    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc" event={"ID":"c205c3a7-9034-44d0-a18c-dab2a9aa4ab0","Type":"ContainerStarted","Data":"10a4401fa6a8ab247f81b94247300a80f1dfc87e4fbfc83aff198acfb188b46d"}
Sep 16 16:55:28.580465 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:28.580387    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:28.789760 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:28.789719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:28.915684 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:28.915654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:29.512575 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:29.512515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:29.626960 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:29.626938    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"
Sep 16 16:55:29.706611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:29.706583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c205c3a7-9034-44d0-a18c-dab2a9aa4ab0-maas-api-service-ca\") pod \"c205c3a7-9034-44d0-a18c-dab2a9aa4ab0\" (UID: \"c205c3a7-9034-44d0-a18c-dab2a9aa4ab0\") "
Sep 16 16:55:29.706934 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:29.706909    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c205c3a7-9034-44d0-a18c-dab2a9aa4ab0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c205c3a7-9034-44d0-a18c-dab2a9aa4ab0" (UID: "c205c3a7-9034-44d0-a18c-dab2a9aa4ab0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:29.718743 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:29.718695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:29.808024 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:29.807943    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c205c3a7-9034-44d0-a18c-dab2a9aa4ab0-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:30.292038 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:30.291999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:30.501241 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:30.501211    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc" event={"ID":"c205c3a7-9034-44d0-a18c-dab2a9aa4ab0","Type":"ContainerDied","Data":"10a4401fa6a8ab247f81b94247300a80f1dfc87e4fbfc83aff198acfb188b46d"}
Sep 16 16:55:30.501241 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:30.501232    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"
Sep 16 16:55:30.501502 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:30.501255    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10a4401fa6a8ab247f81b94247300a80f1dfc87e4fbfc83aff198acfb188b46d"
Sep 16 16:55:31.112355 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:31.112315    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:31.867175 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:31.867137    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:31.940190 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:31.940148    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:34.882764 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.882732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:34.982290 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.982261    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"]
Sep 16 16:55:34.982679 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.982665    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c205c3a7-9034-44d0-a18c-dab2a9aa4ab0" containerName="revoke-keys"
Sep 16 16:55:34.982730 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.982682    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c205c3a7-9034-44d0-a18c-dab2a9aa4ab0" containerName="revoke-keys"
Sep 16 16:55:34.982812 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.982764    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c205c3a7-9034-44d0-a18c-dab2a9aa4ab0" containerName="revoke-keys"
Sep 16 16:55:34.985944 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.985927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"
Sep 16 16:55:34.988207 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.988185    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:55:34.988207 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.988202    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:55:34.993902 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:34.993876    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"]
Sep 16 16:55:35.055597 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:35.055563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91dd6f15-a784-4bc3-bd53-ac6caa563608-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz\" (UID: \"91dd6f15-a784-4bc3-bd53-ac6caa563608\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"
Sep 16 16:55:35.099379 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:35.099346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:35.156707 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:35.156680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91dd6f15-a784-4bc3-bd53-ac6caa563608-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz\" (UID: \"91dd6f15-a784-4bc3-bd53-ac6caa563608\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"
Sep 16 16:55:35.157307 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:35.157284    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91dd6f15-a784-4bc3-bd53-ac6caa563608-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz\" (UID: \"91dd6f15-a784-4bc3-bd53-ac6caa563608\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"
Sep 16 16:55:35.299059 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:35.299022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"
Sep 16 16:55:35.575054 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:35.575028    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"]
Sep 16 16:55:35.576377 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:55:35.576353    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91dd6f15_a784_4bc3_bd53_ac6caa563608.slice/crio-5ffb4ec8cd37283e41b3c9e23c896760de81e430f829e1382d7248be5c6ab1b7 WatchSource:0}: Error finding container 5ffb4ec8cd37283e41b3c9e23c896760de81e430f829e1382d7248be5c6ab1b7: Status 404 returned error can't find the container with id 5ffb4ec8cd37283e41b3c9e23c896760de81e430f829e1382d7248be5c6ab1b7
Sep 16 16:55:35.844595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:35.844512    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:36.086550 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:36.086496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:36.537666 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:36.537632    2577 generic.go:356] "Generic (PLEG): container finished" podID="91dd6f15-a784-4bc3-bd53-ac6caa563608" containerID="1bc4d12558810489630e21fb2de4a02e29e49a15cc9fcb3776aba743c9808ea3" exitCode=0
Sep 16 16:55:36.537856 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:36.537727    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz" event={"ID":"91dd6f15-a784-4bc3-bd53-ac6caa563608","Type":"ContainerDied","Data":"1bc4d12558810489630e21fb2de4a02e29e49a15cc9fcb3776aba743c9808ea3"}
Sep 16 16:55:36.537856 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:36.537790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz" event={"ID":"91dd6f15-a784-4bc3-bd53-ac6caa563608","Type":"ContainerStarted","Data":"5ffb4ec8cd37283e41b3c9e23c896760de81e430f829e1382d7248be5c6ab1b7"}
Sep 16 16:55:36.902826 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:36.902780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:37.487946 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.487909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:37.524851 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.524821    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"]
Sep 16 16:55:37.525060 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.525036    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9" podUID="f02a3453-e414-4683-b3d9-245070c0c33c" containerName="payload-processing" containerID="cri-o://2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd" gracePeriod=30
Sep 16 16:55:37.534556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.534511    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"]
Sep 16 16:55:37.534939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.534909    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf" podUID="3875cd77-6d4c-4809-b377-17f7bae294ca" containerName="payload-pre-processing" containerID="cri-o://9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d" gracePeriod=30
Sep 16 16:55:37.587516 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.587482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:37.700353 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.700325    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"
Sep 16 16:55:37.711961 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.711930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:37.781349 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.781319    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91dd6f15-a784-4bc3-bd53-ac6caa563608-maas-api-service-ca\") pod \"91dd6f15-a784-4bc3-bd53-ac6caa563608\" (UID: \"91dd6f15-a784-4bc3-bd53-ac6caa563608\") "
Sep 16 16:55:37.781713 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.781686    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91dd6f15-a784-4bc3-bd53-ac6caa563608-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91dd6f15-a784-4bc3-bd53-ac6caa563608" (UID: "91dd6f15-a784-4bc3-bd53-ac6caa563608"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:37.804495 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.804472    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:55:37.807482 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.807464    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:55:37.878695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.878660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:37.882436 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.882406    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bj9q\" (UniqueName: \"kubernetes.io/projected/3875cd77-6d4c-4809-b377-17f7bae294ca-kube-api-access-8bj9q\") pod \"3875cd77-6d4c-4809-b377-17f7bae294ca\" (UID: \"3875cd77-6d4c-4809-b377-17f7bae294ca\") "
Sep 16 16:55:37.882609 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.882449    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3875cd77-6d4c-4809-b377-17f7bae294ca-plugins-config-volume\") pod \"3875cd77-6d4c-4809-b377-17f7bae294ca\" (UID: \"3875cd77-6d4c-4809-b377-17f7bae294ca\") "
Sep 16 16:55:37.882609 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.882583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f02a3453-e414-4683-b3d9-245070c0c33c-plugins-config-volume\") pod \"f02a3453-e414-4683-b3d9-245070c0c33c\" (UID: \"f02a3453-e414-4683-b3d9-245070c0c33c\") "
Sep 16 16:55:37.882757 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.882651    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrs6f\" (UniqueName: \"kubernetes.io/projected/f02a3453-e414-4683-b3d9-245070c0c33c-kube-api-access-hrs6f\") pod \"f02a3453-e414-4683-b3d9-245070c0c33c\" (UID: \"f02a3453-e414-4683-b3d9-245070c0c33c\") "
Sep 16 16:55:37.883184 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.882901    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3875cd77-6d4c-4809-b377-17f7bae294ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3875cd77-6d4c-4809-b377-17f7bae294ca" (UID: "3875cd77-6d4c-4809-b377-17f7bae294ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:37.883184 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.882977    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f02a3453-e414-4683-b3d9-245070c0c33c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f02a3453-e414-4683-b3d9-245070c0c33c" (UID: "f02a3453-e414-4683-b3d9-245070c0c33c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:37.883184 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.882990    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91dd6f15-a784-4bc3-bd53-ac6caa563608-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:37.883184 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.883008    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3875cd77-6d4c-4809-b377-17f7bae294ca-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:37.884778 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.884752    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3875cd77-6d4c-4809-b377-17f7bae294ca-kube-api-access-8bj9q" (OuterVolumeSpecName: "kube-api-access-8bj9q") pod "3875cd77-6d4c-4809-b377-17f7bae294ca" (UID: "3875cd77-6d4c-4809-b377-17f7bae294ca"). InnerVolumeSpecName "kube-api-access-8bj9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:37.885042 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.885019    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f02a3453-e414-4683-b3d9-245070c0c33c-kube-api-access-hrs6f" (OuterVolumeSpecName: "kube-api-access-hrs6f") pod "f02a3453-e414-4683-b3d9-245070c0c33c" (UID: "f02a3453-e414-4683-b3d9-245070c0c33c"). InnerVolumeSpecName "kube-api-access-hrs6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:37.909689 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.909655    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"]
Sep 16 16:55:37.917642 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.917615    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-eb2ab2-rk6lc"]
Sep 16 16:55:37.972299 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.972252    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:37.984463 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.984426    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f02a3453-e414-4683-b3d9-245070c0c33c-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:37.984463 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.984463    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrs6f\" (UniqueName: \"kubernetes.io/projected/f02a3453-e414-4683-b3d9-245070c0c33c-kube-api-access-hrs6f\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:37.984463 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:37.984473    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bj9q\" (UniqueName: \"kubernetes.io/projected/3875cd77-6d4c-4809-b377-17f7bae294ca-kube-api-access-8bj9q\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:38.187872 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.187836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:38.551856 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.551828    2577 generic.go:356] "Generic (PLEG): container finished" podID="3875cd77-6d4c-4809-b377-17f7bae294ca" containerID="9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d" exitCode=0
Sep 16 16:55:38.552263 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.551874    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"
Sep 16 16:55:38.552263 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.551915    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf" event={"ID":"3875cd77-6d4c-4809-b377-17f7bae294ca","Type":"ContainerDied","Data":"9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d"}
Sep 16 16:55:38.552263 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.551966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf" event={"ID":"3875cd77-6d4c-4809-b377-17f7bae294ca","Type":"ContainerDied","Data":"80bcff005cc68b41498a66a2e304baf66b6f8735b526add5137143e4908a6750"}
Sep 16 16:55:38.552263 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.551991    2577 scope.go:117] "RemoveContainer" containerID="9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d"
Sep 16 16:55:38.553244 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.553225    2577 generic.go:356] "Generic (PLEG): container finished" podID="f02a3453-e414-4683-b3d9-245070c0c33c" containerID="2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd" exitCode=0
Sep 16 16:55:38.553338 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.553268    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"
Sep 16 16:55:38.553338 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.553303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9" event={"ID":"f02a3453-e414-4683-b3d9-245070c0c33c","Type":"ContainerDied","Data":"2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd"}
Sep 16 16:55:38.553447 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.553350    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9" event={"ID":"f02a3453-e414-4683-b3d9-245070c0c33c","Type":"ContainerDied","Data":"68b0305fed1d8a4995a77b9ac0e3aa7c71037f6cd70570d6de87277811ecf31a"}
Sep 16 16:55:38.554860 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.554835    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"
Sep 16 16:55:38.554969 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.554834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz" event={"ID":"91dd6f15-a784-4bc3-bd53-ac6caa563608","Type":"ContainerDied","Data":"5ffb4ec8cd37283e41b3c9e23c896760de81e430f829e1382d7248be5c6ab1b7"}
Sep 16 16:55:38.554969 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.554955    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ffb4ec8cd37283e41b3c9e23c896760de81e430f829e1382d7248be5c6ab1b7"
Sep 16 16:55:38.561706 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.561686    2577 scope.go:117] "RemoveContainer" containerID="9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d"
Sep 16 16:55:38.561974 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:55:38.561948    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d\": container with ID starting with 9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d not found: ID does not exist" containerID="9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d"
Sep 16 16:55:38.562051 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.561983    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d"} err="failed to get container status \"9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d\": rpc error: code = NotFound desc = could not find container \"9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d\": container with ID starting with 9bc037ac68ab264d817955b886339592b94c944aef81c0ab2cc4a5e6367d7d5d not found: ID does not exist"
Sep 16 16:55:38.562051 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.562005    2577 scope.go:117] "RemoveContainer" containerID="2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd"
Sep 16 16:55:38.570957 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.570926    2577 scope.go:117] "RemoveContainer" containerID="2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd"
Sep 16 16:55:38.571220 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:55:38.571189    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd\": container with ID starting with 2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd not found: ID does not exist" containerID="2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd"
Sep 16 16:55:38.571306 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.571214    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd"} err="failed to get container status \"2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd\": rpc error: code = NotFound desc = could not find container \"2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd\": container with ID starting with 2b8ee2a799278d8373c8bb1e859fdafba1c493a4f0f86b9f7b17b37fe4669cbd not found: ID does not exist"
Sep 16 16:55:38.583049 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.583029    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"]
Sep 16 16:55:38.587857 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.587837    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-eb2ab2-77cb556dcd-crrrf"]
Sep 16 16:55:38.610401 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.610378    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"]
Sep 16 16:55:38.614664 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.614641    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-eb2ab2-6d6b7d45bc-p49c9"]
Sep 16 16:55:38.772622 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.772584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:38.977355 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.977325    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3875cd77-6d4c-4809-b377-17f7bae294ca" path="/var/lib/kubelet/pods/3875cd77-6d4c-4809-b377-17f7bae294ca/volumes"
Sep 16 16:55:38.977756 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.977741    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c205c3a7-9034-44d0-a18c-dab2a9aa4ab0" path="/var/lib/kubelet/pods/c205c3a7-9034-44d0-a18c-dab2a9aa4ab0/volumes"
Sep 16 16:55:38.978109 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:38.978094    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f02a3453-e414-4683-b3d9-245070c0c33c" path="/var/lib/kubelet/pods/f02a3453-e414-4683-b3d9-245070c0c33c/volumes"
Sep 16 16:55:39.604131 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:39.604094    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:39.960676 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:39.960638    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:40.328125 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:40.328034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:41.170931 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:41.170898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:44.034445 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:44.034409    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:44.134722 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:44.134684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:44.302868 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:44.302775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:44.691305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:44.691274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:45.166545 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:45.166504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:45.890832 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:45.890797    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:45.919293 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:45.919261    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"]
Sep 16 16:55:45.919617 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:45.919585    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n" podUID="a4fdef40-5aa3-4be7-aaa9-46e83b717f30" containerName="payload-processing" containerID="cri-o://456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3" gracePeriod=30
Sep 16 16:55:45.927390 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:45.927357    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"]
Sep 16 16:55:45.927768 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:45.927742    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56" podUID="7727f763-fd42-4c69-8564-ee3e007f69c4" containerName="payload-pre-processing" containerID="cri-o://18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8" gracePeriod=30
Sep 16 16:55:45.991116 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:45.991079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:46.089264 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.089096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:46.118351 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.118318    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"]
Sep 16 16:55:46.120952 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.120916    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-f604b4-q85jz"]
Sep 16 16:55:46.200687 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.200660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:46.202790 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.202767    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:55:46.206164 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.206145    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:55:46.366976 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.366944    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-plugins-config-volume\") pod \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\" (UID: \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\") "
Sep 16 16:55:46.367133 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.367006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8mng\" (UniqueName: \"kubernetes.io/projected/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-kube-api-access-n8mng\") pod \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\" (UID: \"a4fdef40-5aa3-4be7-aaa9-46e83b717f30\") "
Sep 16 16:55:46.367133 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.367031    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7727f763-fd42-4c69-8564-ee3e007f69c4-plugins-config-volume\") pod \"7727f763-fd42-4c69-8564-ee3e007f69c4\" (UID: \"7727f763-fd42-4c69-8564-ee3e007f69c4\") "
Sep 16 16:55:46.367133 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.367107    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kn9j\" (UniqueName: \"kubernetes.io/projected/7727f763-fd42-4c69-8564-ee3e007f69c4-kube-api-access-7kn9j\") pod \"7727f763-fd42-4c69-8564-ee3e007f69c4\" (UID: \"7727f763-fd42-4c69-8564-ee3e007f69c4\") "
Sep 16 16:55:46.367397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.367369    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4fdef40-5aa3-4be7-aaa9-46e83b717f30" (UID: "a4fdef40-5aa3-4be7-aaa9-46e83b717f30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:46.367484 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.367420    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7727f763-fd42-4c69-8564-ee3e007f69c4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7727f763-fd42-4c69-8564-ee3e007f69c4" (UID: "7727f763-fd42-4c69-8564-ee3e007f69c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:55:46.369085 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.369061    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-kube-api-access-n8mng" (OuterVolumeSpecName: "kube-api-access-n8mng") pod "a4fdef40-5aa3-4be7-aaa9-46e83b717f30" (UID: "a4fdef40-5aa3-4be7-aaa9-46e83b717f30"). InnerVolumeSpecName "kube-api-access-n8mng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:46.369181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.369126    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7727f763-fd42-4c69-8564-ee3e007f69c4-kube-api-access-7kn9j" (OuterVolumeSpecName: "kube-api-access-7kn9j") pod "7727f763-fd42-4c69-8564-ee3e007f69c4" (UID: "7727f763-fd42-4c69-8564-ee3e007f69c4"). InnerVolumeSpecName "kube-api-access-7kn9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:55:46.467986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.467894    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kn9j\" (UniqueName: \"kubernetes.io/projected/7727f763-fd42-4c69-8564-ee3e007f69c4-kube-api-access-7kn9j\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:46.467986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.467929    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:46.467986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.467942    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8mng\" (UniqueName: \"kubernetes.io/projected/a4fdef40-5aa3-4be7-aaa9-46e83b717f30-kube-api-access-n8mng\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:46.467986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.467954    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7727f763-fd42-4c69-8564-ee3e007f69c4-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:55:46.485427 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.485394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:46.603841 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.603802    2577 generic.go:356] "Generic (PLEG): container finished" podID="7727f763-fd42-4c69-8564-ee3e007f69c4" containerID="18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8" exitCode=0
Sep 16 16:55:46.604027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.603855    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"
Sep 16 16:55:46.604027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.603889    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56" event={"ID":"7727f763-fd42-4c69-8564-ee3e007f69c4","Type":"ContainerDied","Data":"18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8"}
Sep 16 16:55:46.604027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.603948    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56" event={"ID":"7727f763-fd42-4c69-8564-ee3e007f69c4","Type":"ContainerDied","Data":"72e8a8902a77854e663217e733f9ed03d09e2bbf0f240e45527c7eaff338e529"}
Sep 16 16:55:46.604027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.603979    2577 scope.go:117] "RemoveContainer" containerID="18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8"
Sep 16 16:55:46.605481 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.605454    2577 generic.go:356] "Generic (PLEG): container finished" podID="a4fdef40-5aa3-4be7-aaa9-46e83b717f30" containerID="456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3" exitCode=0
Sep 16 16:55:46.605621 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.605491    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"
Sep 16 16:55:46.605621 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.605558    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n" event={"ID":"a4fdef40-5aa3-4be7-aaa9-46e83b717f30","Type":"ContainerDied","Data":"456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3"}
Sep 16 16:55:46.605717 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.605622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n" event={"ID":"a4fdef40-5aa3-4be7-aaa9-46e83b717f30","Type":"ContainerDied","Data":"ee8cb0c0cb63deb14dcd2819fb8ad361720a2fec90d24e6332bdfa85d1f03169"}
Sep 16 16:55:46.608961 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.608933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:46.614075 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.614050    2577 scope.go:117] "RemoveContainer" containerID="18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8"
Sep 16 16:55:46.614291 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:55:46.614268    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8\": container with ID starting with 18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8 not found: ID does not exist" containerID="18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8"
Sep 16 16:55:46.614374 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.614302    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8"} err="failed to get container status \"18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8\": rpc error: code = NotFound desc = could not find container \"18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8\": container with ID starting with 18ca0cfe42e53426049a3e8bb3560229e3b665814621d6220b0c08fe1b0f1ab8 not found: ID does not exist"
Sep 16 16:55:46.614374 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.614324    2577 scope.go:117] "RemoveContainer" containerID="456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3"
Sep 16 16:55:46.623330 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.623314    2577 scope.go:117] "RemoveContainer" containerID="456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3"
Sep 16 16:55:46.623618 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:55:46.623600    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3\": container with ID starting with 456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3 not found: ID does not exist" containerID="456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3"
Sep 16 16:55:46.623678 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.623625    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3"} err="failed to get container status \"456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3\": rpc error: code = NotFound desc = could not find container \"456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3\": container with ID starting with 456bd738d09a7abb45e634d343765c54e99fbb6a00699124db88fcb0e3fc74b3 not found: ID does not exist"
Sep 16 16:55:46.629654 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.629633    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"]
Sep 16 16:55:46.634506 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.634487    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-f604b4-8955b4bc4-zhc56"]
Sep 16 16:55:46.648633 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.648611    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"]
Sep 16 16:55:46.653815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.653797    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-f604b4-854569ddc-qns9n"]
Sep 16 16:55:46.977249 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.977221    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7727f763-fd42-4c69-8564-ee3e007f69c4" path="/var/lib/kubelet/pods/7727f763-fd42-4c69-8564-ee3e007f69c4/volumes"
Sep 16 16:55:46.977579 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.977565    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91dd6f15-a784-4bc3-bd53-ac6caa563608" path="/var/lib/kubelet/pods/91dd6f15-a784-4bc3-bd53-ac6caa563608/volumes"
Sep 16 16:55:46.977898 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:46.977886    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4fdef40-5aa3-4be7-aaa9-46e83b717f30" path="/var/lib/kubelet/pods/a4fdef40-5aa3-4be7-aaa9-46e83b717f30/volumes"
Sep 16 16:55:47.533494 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:47.533458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:48.691293 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:48.691262    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:50.412393 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:50.412357    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:50.529450 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:50.529421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:50.622378 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:50.622350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:52.603088 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:52.603042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:52.740789 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:52.740749    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:55:52.927493 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:55:52.927453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:00.232066 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:00.232019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:01.068182 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:01.068146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:01.790013 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:01.789972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:06.864702 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:06.864662    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:07.055556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:07.055504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:07.490097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:07.490058    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:07.808242 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:07.808159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:07.976719 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:07.976679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:08.580304 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:08.580268    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:08.688607 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:08.688574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:08.793039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:08.793001    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:08.939831 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:08.939797    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:09.415325 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:09.415293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:09.516067 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:09.516030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:09.616111 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:09.616073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:10.055100 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:10.055064    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:10.629028 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:10.628974    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:11.094829 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:11.094761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:11.503147 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:11.503103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:11.602055 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:11.602021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:12.489745 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:12.489708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:13.629595 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:13.629558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:13.730048 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:13.730004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:14.207347 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.207308    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:14.308484 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.308451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:14.331275 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331246    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"]
Sep 16 16:56:14.331658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331644    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3875cd77-6d4c-4809-b377-17f7bae294ca" containerName="payload-pre-processing"
Sep 16 16:56:14.331720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331660    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3875cd77-6d4c-4809-b377-17f7bae294ca" containerName="payload-pre-processing"
Sep 16 16:56:14.331720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331670    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f02a3453-e414-4683-b3d9-245070c0c33c" containerName="payload-processing"
Sep 16 16:56:14.331720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331676    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f02a3453-e414-4683-b3d9-245070c0c33c" containerName="payload-processing"
Sep 16 16:56:14.331720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331691    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91dd6f15-a784-4bc3-bd53-ac6caa563608" containerName="revoke-keys"
Sep 16 16:56:14.331720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331697    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="91dd6f15-a784-4bc3-bd53-ac6caa563608" containerName="revoke-keys"
Sep 16 16:56:14.331720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331706    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4fdef40-5aa3-4be7-aaa9-46e83b717f30" containerName="payload-processing"
Sep 16 16:56:14.331720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331711    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4fdef40-5aa3-4be7-aaa9-46e83b717f30" containerName="payload-processing"
Sep 16 16:56:14.331947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331723    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7727f763-fd42-4c69-8564-ee3e007f69c4" containerName="payload-pre-processing"
Sep 16 16:56:14.331947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331729    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7727f763-fd42-4c69-8564-ee3e007f69c4" containerName="payload-pre-processing"
Sep 16 16:56:14.331947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331797    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7727f763-fd42-4c69-8564-ee3e007f69c4" containerName="payload-pre-processing"
Sep 16 16:56:14.331947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331806    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4fdef40-5aa3-4be7-aaa9-46e83b717f30" containerName="payload-processing"
Sep 16 16:56:14.331947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331812    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="91dd6f15-a784-4bc3-bd53-ac6caa563608" containerName="revoke-keys"
Sep 16 16:56:14.331947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331818    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f02a3453-e414-4683-b3d9-245070c0c33c" containerName="payload-processing"
Sep 16 16:56:14.331947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.331826    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3875cd77-6d4c-4809-b377-17f7bae294ca" containerName="payload-pre-processing"
Sep 16 16:56:14.335006 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.334991    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:14.337645 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.337624    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-97b645\""
Sep 16 16:56:14.337742 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.337699    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-97b645-dockercfg-ck62n\""
Sep 16 16:56:14.347799 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.347776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"]
Sep 16 16:56:14.381192 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.381163    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"]
Sep 16 16:56:14.384582 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.384566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:14.394986 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.394963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"]
Sep 16 16:56:14.406991 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.406959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d6cdf6-8400-4f62-a729-d8738fb88d06-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs\" (UID: \"43d6cdf6-8400-4f62-a729-d8738fb88d06\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:14.407128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.407109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4lxh\" (UniqueName: \"kubernetes.io/projected/b1a6d3c8-3028-484f-adb6-efbcc62c928c-kube-api-access-d4lxh\") pod \"payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh\" (UID: \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\") " pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:14.407227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.407145    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a6d3c8-3028-484f-adb6-efbcc62c928c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh\" (UID: \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\") " pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:14.407227 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.407213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n49kd\" (UniqueName: \"kubernetes.io/projected/43d6cdf6-8400-4f62-a729-d8738fb88d06-kube-api-access-n49kd\") pod \"payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs\" (UID: \"43d6cdf6-8400-4f62-a729-d8738fb88d06\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:14.508655 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.508556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n49kd\" (UniqueName: \"kubernetes.io/projected/43d6cdf6-8400-4f62-a729-d8738fb88d06-kube-api-access-n49kd\") pod \"payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs\" (UID: \"43d6cdf6-8400-4f62-a729-d8738fb88d06\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:14.508655 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.508648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d6cdf6-8400-4f62-a729-d8738fb88d06-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs\" (UID: \"43d6cdf6-8400-4f62-a729-d8738fb88d06\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:14.508891 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.508761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4lxh\" (UniqueName: \"kubernetes.io/projected/b1a6d3c8-3028-484f-adb6-efbcc62c928c-kube-api-access-d4lxh\") pod \"payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh\" (UID: \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\") " pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:14.508891 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.508787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a6d3c8-3028-484f-adb6-efbcc62c928c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh\" (UID: \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\") " pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:14.509471 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.509447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a6d3c8-3028-484f-adb6-efbcc62c928c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh\" (UID: \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\") " pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:14.509635 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.509613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d6cdf6-8400-4f62-a729-d8738fb88d06-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs\" (UID: \"43d6cdf6-8400-4f62-a729-d8738fb88d06\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:14.517205 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.517184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n49kd\" (UniqueName: \"kubernetes.io/projected/43d6cdf6-8400-4f62-a729-d8738fb88d06-kube-api-access-n49kd\") pod \"payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs\" (UID: \"43d6cdf6-8400-4f62-a729-d8738fb88d06\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:14.517300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.517225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4lxh\" (UniqueName: \"kubernetes.io/projected/b1a6d3c8-3028-484f-adb6-efbcc62c928c-kube-api-access-d4lxh\") pod \"payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh\" (UID: \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\") " pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:14.645477 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.645447    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:14.695197 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.694884    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:14.919256 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.919230    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"]
Sep 16 16:56:14.921741 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:56:14.921717    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a6d3c8_3028_484f_adb6_efbcc62c928c.slice/crio-87a37c57d82d437f4b059380e81583cf0e06f287b7ff7a75097f9b01e9d7c4d5 WatchSource:0}: Error finding container 87a37c57d82d437f4b059380e81583cf0e06f287b7ff7a75097f9b01e9d7c4d5: Status 404 returned error can't find the container with id 87a37c57d82d437f4b059380e81583cf0e06f287b7ff7a75097f9b01e9d7c4d5
Sep 16 16:56:14.979128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:14.979102    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"]
Sep 16 16:56:14.981719 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:56:14.981687    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d6cdf6_8400_4f62_a729_d8738fb88d06.slice/crio-53120a45f77475c893db9b73ad17738603eddcc6fa990ee17caa63ba5d93217f WatchSource:0}: Error finding container 53120a45f77475c893db9b73ad17738603eddcc6fa990ee17caa63ba5d93217f: Status 404 returned error can't find the container with id 53120a45f77475c893db9b73ad17738603eddcc6fa990ee17caa63ba5d93217f
Sep 16 16:56:15.785604 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:15.785563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh" event={"ID":"b1a6d3c8-3028-484f-adb6-efbcc62c928c","Type":"ContainerStarted","Data":"a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d"}
Sep 16 16:56:15.786059 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:15.785670    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:15.786059 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:15.785699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh" event={"ID":"b1a6d3c8-3028-484f-adb6-efbcc62c928c","Type":"ContainerStarted","Data":"87a37c57d82d437f4b059380e81583cf0e06f287b7ff7a75097f9b01e9d7c4d5"}
Sep 16 16:56:15.787232 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:15.787207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs" event={"ID":"43d6cdf6-8400-4f62-a729-d8738fb88d06","Type":"ContainerStarted","Data":"13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c"}
Sep 16 16:56:15.787335 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:15.787246    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs" event={"ID":"43d6cdf6-8400-4f62-a729-d8738fb88d06","Type":"ContainerStarted","Data":"53120a45f77475c893db9b73ad17738603eddcc6fa990ee17caa63ba5d93217f"}
Sep 16 16:56:15.787335 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:15.787266    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:15.806277 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:15.806230    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh" podStartSLOduration=1.806217167 podStartE2EDuration="1.806217167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:56:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:56:15.804629422 +0000 UTC m=+1191.301502547" watchObservedRunningTime="2026-09-16 16:56:15.806217167 +0000 UTC m=+1191.303090290"
Sep 16 16:56:15.824507 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:15.824464    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs" podStartSLOduration=1.824450728 podStartE2EDuration="1.824450728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:56:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:56:15.82229505 +0000 UTC m=+1191.319168187" watchObservedRunningTime="2026-09-16 16:56:15.824450728 +0000 UTC m=+1191.321323851"
Sep 16 16:56:16.538598 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:16.538561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:16.639429 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:16.639391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:16.904585 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:16.904548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:17.947583 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:17.947547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:18.046708 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:18.046666    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:18.257226 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:18.257148    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:18.592702 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:18.592617    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:24.258545 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:24.258479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:24.357840 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:24.357811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:24.756512 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:24.756476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:24.830082 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:24.830051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:24.957393 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:24.957282    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:56:24.976726 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:24.961029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 16:56:25.065296 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:25.065271    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:25.298983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:25.298950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:25.567810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:25.567767    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:26.796348 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:26.796316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:56:26.796873 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:26.796401    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:56:30.964777 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:30.964738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:31.623104 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:31.623061    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:31.714509 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:31.714472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:31.817759 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:31.817727    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:32.765631 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:32.765591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:33.690089 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:33.690052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:34.414677 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:34.414645    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:34.519554 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:34.519495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:34.619446 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:34.619411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:34.901918 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:34.901890    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:34.975238 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:34.975205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:35.014673 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.014646    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"]
Sep 16 16:56:35.019693 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.019677    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.022224 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.022200    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f3a3973e\""
Sep 16 16:56:35.022354 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.022241    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f3a3973e-dockercfg-rwcsk\""
Sep 16 16:56:35.032932 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.032909    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"]
Sep 16 16:56:35.042052 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.042026    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"]
Sep 16 16:56:35.045224 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.045205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.061617 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.061594    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"]
Sep 16 16:56:35.102188 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.102169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r2fx\" (UniqueName: \"kubernetes.io/projected/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-kube-api-access-5r2fx\") pod \"payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd\" (UID: \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.102299 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.102232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd\" (UID: \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.202862 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.202777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd\" (UID: \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.203033 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.202893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-plugins-config-volume\") pod \"payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9\" (UID: \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.203033 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.202928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r2fx\" (UniqueName: \"kubernetes.io/projected/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-kube-api-access-5r2fx\") pod \"payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd\" (UID: \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.203033 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.202961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4h4s\" (UniqueName: \"kubernetes.io/projected/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-kube-api-access-f4h4s\") pod \"payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9\" (UID: \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.203520 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.203499    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd\" (UID: \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.212037 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.212011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r2fx\" (UniqueName: \"kubernetes.io/projected/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-kube-api-access-5r2fx\") pod \"payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd\" (UID: \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.303772 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.303735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-plugins-config-volume\") pod \"payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9\" (UID: \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.303772 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.303782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4h4s\" (UniqueName: \"kubernetes.io/projected/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-kube-api-access-f4h4s\") pod \"payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9\" (UID: \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.304512 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.304487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-plugins-config-volume\") pod \"payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9\" (UID: \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.315496 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.315458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4h4s\" (UniqueName: \"kubernetes.io/projected/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-kube-api-access-f4h4s\") pod \"payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9\" (UID: \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\") " pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.330214 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.330181    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.355257 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.355230    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.382651 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.382210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:35.611585 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.611555    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"]
Sep 16 16:56:35.612204 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:56:35.612176    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2e815f_78bb_4ab1_9cfe_3d1a2134f8c0.slice/crio-446ee7866014a0e6ebf44b5065d1182a1d00787e15f331f8d7f7e8e263d07a5b WatchSource:0}: Error finding container 446ee7866014a0e6ebf44b5065d1182a1d00787e15f331f8d7f7e8e263d07a5b: Status 404 returned error can't find the container with id 446ee7866014a0e6ebf44b5065d1182a1d00787e15f331f8d7f7e8e263d07a5b
Sep 16 16:56:35.650574 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.650552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"]
Sep 16 16:56:35.651436 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:56:35.651414    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09dd0d1b_0bdf_454a_aa6f_b9e0b9c6c00e.slice/crio-b246e7901ef25d7973dda211b0f13537736499cbe63105f59627d02c8615ab68 WatchSource:0}: Error finding container b246e7901ef25d7973dda211b0f13537736499cbe63105f59627d02c8615ab68: Status 404 returned error can't find the container with id b246e7901ef25d7973dda211b0f13537736499cbe63105f59627d02c8615ab68
Sep 16 16:56:35.906703 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.906661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd" event={"ID":"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0","Type":"ContainerStarted","Data":"62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026"}
Sep 16 16:56:35.906900 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.906755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd" event={"ID":"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0","Type":"ContainerStarted","Data":"446ee7866014a0e6ebf44b5065d1182a1d00787e15f331f8d7f7e8e263d07a5b"}
Sep 16 16:56:35.906900 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.906878    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:35.908390 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.908355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9" event={"ID":"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e","Type":"ContainerStarted","Data":"14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9"}
Sep 16 16:56:35.908497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.908405    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9" event={"ID":"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e","Type":"ContainerStarted","Data":"b246e7901ef25d7973dda211b0f13537736499cbe63105f59627d02c8615ab68"}
Sep 16 16:56:35.908497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.908446    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:35.924239 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.924189    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd" podStartSLOduration=1.924173411 podStartE2EDuration="1.924173411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:56:35.923696623 +0000 UTC m=+1211.420569750" watchObservedRunningTime="2026-09-16 16:56:35.924173411 +0000 UTC m=+1211.421046535"
Sep 16 16:56:35.942411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:35.942360    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9" podStartSLOduration=0.9423446 podStartE2EDuration="942.3446ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:56:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:56:35.939907988 +0000 UTC m=+1211.436781105" watchObservedRunningTime="2026-09-16 16:56:35.9423446 +0000 UTC m=+1211.439217728"
Sep 16 16:56:36.189373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:36.189276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:36.288500 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:36.288469    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:37.174611 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:37.174571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:37.929546 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:37.929498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:46.915235 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:46.915204    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:56:46.915737 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:46.915269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:56:56.611240 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:56.611198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:56.818313 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:56.818270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:56.995815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:56.995786    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:57.348786 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:57.348681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:57.489435 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:57.489403    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:57.777111 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:57.777078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:58.174930 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:58.174901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:59.383808 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.383773    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:59.402431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.402398    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"]
Sep 16 16:56:59.411543 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.411494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"
Sep 16 16:56:59.416148 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.416116    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:56:59.420992 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.420975    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:56:59.426822 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.426798    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"]
Sep 16 16:56:59.525592 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.525545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa32618a-8a77-4461-b164-d645e8a0dba7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-97b645-rsvks\" (UID: \"fa32618a-8a77-4461-b164-d645e8a0dba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"
Sep 16 16:56:59.626372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.626328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa32618a-8a77-4461-b164-d645e8a0dba7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-97b645-rsvks\" (UID: \"fa32618a-8a77-4461-b164-d645e8a0dba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"
Sep 16 16:56:59.626980 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.626959    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa32618a-8a77-4461-b164-d645e8a0dba7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-97b645-rsvks\" (UID: \"fa32618a-8a77-4461-b164-d645e8a0dba7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"
Sep 16 16:56:59.722765 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.722731    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"
Sep 16 16:56:59.764938 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.764899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:56:59.996013 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:56:59.995989    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"]
Sep 16 16:56:59.997361 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:56:59.997334    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa32618a_8a77_4461_b164_d645e8a0dba7.slice/crio-c8d6086c5983ffd053a663b220a609f2ac8d8190e5ab459a525e9008613ea0bb WatchSource:0}: Error finding container c8d6086c5983ffd053a663b220a609f2ac8d8190e5ab459a525e9008613ea0bb: Status 404 returned error can't find the container with id c8d6086c5983ffd053a663b220a609f2ac8d8190e5ab459a525e9008613ea0bb
Sep 16 16:57:00.047478 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:00.047446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks" event={"ID":"fa32618a-8a77-4461-b164-d645e8a0dba7","Type":"ContainerStarted","Data":"c8d6086c5983ffd053a663b220a609f2ac8d8190e5ab459a525e9008613ea0bb"}
Sep 16 16:57:00.706518 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:00.706486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:01.054624 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:01.054511    2577 generic.go:356] "Generic (PLEG): container finished" podID="fa32618a-8a77-4461-b164-d645e8a0dba7" containerID="37514886315e7c042b4367606a32ad03f10fbf8c7b82e6b935e4528196efab03" exitCode=0
Sep 16 16:57:01.054624 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:01.054598    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks" event={"ID":"fa32618a-8a77-4461-b164-d645e8a0dba7","Type":"ContainerDied","Data":"37514886315e7c042b4367606a32ad03f10fbf8c7b82e6b935e4528196efab03"}
Sep 16 16:57:01.458848 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:01.458788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:02.196224 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:02.196193    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"
Sep 16 16:57:02.248651 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:02.248617    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa32618a-8a77-4461-b164-d645e8a0dba7-maas-api-service-ca\") pod \"fa32618a-8a77-4461-b164-d645e8a0dba7\" (UID: \"fa32618a-8a77-4461-b164-d645e8a0dba7\") "
Sep 16 16:57:02.249007 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:02.248983    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa32618a-8a77-4461-b164-d645e8a0dba7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa32618a-8a77-4461-b164-d645e8a0dba7" (UID: "fa32618a-8a77-4461-b164-d645e8a0dba7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:02.267968 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:02.267932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:02.349673 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:02.349606    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa32618a-8a77-4461-b164-d645e8a0dba7-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:03.068350 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:03.068328    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"
Sep 16 16:57:03.068579 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:03.068325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks" event={"ID":"fa32618a-8a77-4461-b164-d645e8a0dba7","Type":"ContainerDied","Data":"c8d6086c5983ffd053a663b220a609f2ac8d8190e5ab459a525e9008613ea0bb"}
Sep 16 16:57:03.068579 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:03.068456    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8d6086c5983ffd053a663b220a609f2ac8d8190e5ab459a525e9008613ea0bb"
Sep 16 16:57:03.078563 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:03.078522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:04.103697 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.103659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:04.581668 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.581635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:04.684107 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.684065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:04.911766 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.911729    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"]
Sep 16 16:57:04.912165 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.912152    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa32618a-8a77-4461-b164-d645e8a0dba7" containerName="revoke-keys"
Sep 16 16:57:04.912213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.912167    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa32618a-8a77-4461-b164-d645e8a0dba7" containerName="revoke-keys"
Sep 16 16:57:04.912253 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.912230    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa32618a-8a77-4461-b164-d645e8a0dba7" containerName="revoke-keys"
Sep 16 16:57:04.915183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.915160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"
Sep 16 16:57:04.917949 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.917932    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:57:04.918251 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.918238    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:57:04.926644 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.926469    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"]
Sep 16 16:57:04.971997 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:04.971962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c698c4-8623-4765-8116-6fd42e0cc888-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm\" (UID: \"b7c698c4-8623-4765-8116-6fd42e0cc888\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"
Sep 16 16:57:05.072921 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:05.072887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c698c4-8623-4765-8116-6fd42e0cc888-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm\" (UID: \"b7c698c4-8623-4765-8116-6fd42e0cc888\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"
Sep 16 16:57:05.073510 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:05.073489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c698c4-8623-4765-8116-6fd42e0cc888-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm\" (UID: \"b7c698c4-8623-4765-8116-6fd42e0cc888\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"
Sep 16 16:57:05.090865 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:05.090831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:05.228838 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:05.228750    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"
Sep 16 16:57:05.505973 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:05.505944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"]
Sep 16 16:57:05.507523 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:57:05.507490    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c698c4_8623_4765_8116_6fd42e0cc888.slice/crio-da3565325e06ec10582106b00776f302b8c2f2b49a33fe843368b7ad5e25aaf9 WatchSource:0}: Error finding container da3565325e06ec10582106b00776f302b8c2f2b49a33fe843368b7ad5e25aaf9: Status 404 returned error can't find the container with id da3565325e06ec10582106b00776f302b8c2f2b49a33fe843368b7ad5e25aaf9
Sep 16 16:57:05.985610 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:05.985574    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:06.089208 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:06.089118    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm" event={"ID":"b7c698c4-8623-4765-8116-6fd42e0cc888","Type":"ContainerStarted","Data":"8606c6a214311a7c22077f43666858776d2dde80d4aa74f4a87d924bb10bb12a"}
Sep 16 16:57:06.089208 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:06.089172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm" event={"ID":"b7c698c4-8623-4765-8116-6fd42e0cc888","Type":"ContainerStarted","Data":"da3565325e06ec10582106b00776f302b8c2f2b49a33fe843368b7ad5e25aaf9"}
Sep 16 16:57:06.110332 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:06.110283    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm" podStartSLOduration=2.11026722 podStartE2EDuration="2.11026722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:06.108564104 +0000 UTC m=+1241.605437230" watchObservedRunningTime="2026-09-16 16:57:06.11026722 +0000 UTC m=+1241.607140344"
Sep 16 16:57:06.795726 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:06.795691    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:07.097106 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:07.097020    2577 generic.go:356] "Generic (PLEG): container finished" podID="b7c698c4-8623-4765-8116-6fd42e0cc888" containerID="8606c6a214311a7c22077f43666858776d2dde80d4aa74f4a87d924bb10bb12a" exitCode=0
Sep 16 16:57:07.097268 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:07.097111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm" event={"ID":"b7c698c4-8623-4765-8116-6fd42e0cc888","Type":"ContainerDied","Data":"8606c6a214311a7c22077f43666858776d2dde80d4aa74f4a87d924bb10bb12a"}
Sep 16 16:57:07.719880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:07.719840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:08.238580 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:08.238555    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"
Sep 16 16:57:08.304295 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:08.304270    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c698c4-8623-4765-8116-6fd42e0cc888-maas-api-service-ca\") pod \"b7c698c4-8623-4765-8116-6fd42e0cc888\" (UID: \"b7c698c4-8623-4765-8116-6fd42e0cc888\") "
Sep 16 16:57:08.304672 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:08.304640    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7c698c4-8623-4765-8116-6fd42e0cc888-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7c698c4-8623-4765-8116-6fd42e0cc888" (UID: "b7c698c4-8623-4765-8116-6fd42e0cc888"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:08.405713 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:08.405685    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7c698c4-8623-4765-8116-6fd42e0cc888-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:08.620300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:08.620269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:09.022983 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:09.022950    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:09.110988 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:09.110951    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"
Sep 16 16:57:09.111160 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:09.110997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm" event={"ID":"b7c698c4-8623-4765-8116-6fd42e0cc888","Type":"ContainerDied","Data":"da3565325e06ec10582106b00776f302b8c2f2b49a33fe843368b7ad5e25aaf9"}
Sep 16 16:57:09.111160 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:09.111057    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da3565325e06ec10582106b00776f302b8c2f2b49a33fe843368b7ad5e25aaf9"
Sep 16 16:57:09.412411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:09.412375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:10.173810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.173769    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:10.210947 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.210915    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"]
Sep 16 16:57:10.211341 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.211328    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7c698c4-8623-4765-8116-6fd42e0cc888" containerName="revoke-keys"
Sep 16 16:57:10.211394 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.211342    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7c698c4-8623-4765-8116-6fd42e0cc888" containerName="revoke-keys"
Sep 16 16:57:10.211439 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.211415    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7c698c4-8623-4765-8116-6fd42e0cc888" containerName="revoke-keys"
Sep 16 16:57:10.214373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.214356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"
Sep 16 16:57:10.217664 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.217645    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:57:10.217863 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.217846    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:57:10.229471 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.229448    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"]
Sep 16 16:57:10.322061 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.322029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6468f446-19c1-4cb7-80fb-d6fae7971518-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h\" (UID: \"6468f446-19c1-4cb7-80fb-d6fae7971518\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"
Sep 16 16:57:10.337810 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.337775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:10.368426 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.368395    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"]
Sep 16 16:57:10.368646 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.368619    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh" podUID="b1a6d3c8-3028-484f-adb6-efbcc62c928c" containerName="payload-processing" containerID="cri-o://a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d" gracePeriod=30
Sep 16 16:57:10.375602 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.375577    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"]
Sep 16 16:57:10.375841 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.375753    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs" podUID="43d6cdf6-8400-4f62-a729-d8738fb88d06" containerName="payload-pre-processing" containerID="cri-o://13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c" gracePeriod=30
Sep 16 16:57:10.423221 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.423193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6468f446-19c1-4cb7-80fb-d6fae7971518-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h\" (UID: \"6468f446-19c1-4cb7-80fb-d6fae7971518\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"
Sep 16 16:57:10.424027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.423966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6468f446-19c1-4cb7-80fb-d6fae7971518-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h\" (UID: \"6468f446-19c1-4cb7-80fb-d6fae7971518\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"
Sep 16 16:57:10.424206 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.424181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:10.524242 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.524204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"
Sep 16 16:57:10.524429 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.524253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:10.610511 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.610477    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:10.643372 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.643345    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:57:10.664589 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.664565    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:57:10.682634 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.682561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:10.725825 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.725797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d6cdf6-8400-4f62-a729-d8738fb88d06-plugins-config-volume\") pod \"43d6cdf6-8400-4f62-a729-d8738fb88d06\" (UID: \"43d6cdf6-8400-4f62-a729-d8738fb88d06\") "
Sep 16 16:57:10.725981 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.725842    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4lxh\" (UniqueName: \"kubernetes.io/projected/b1a6d3c8-3028-484f-adb6-efbcc62c928c-kube-api-access-d4lxh\") pod \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\" (UID: \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\") "
Sep 16 16:57:10.725981 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.725875    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a6d3c8-3028-484f-adb6-efbcc62c928c-plugins-config-volume\") pod \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\" (UID: \"b1a6d3c8-3028-484f-adb6-efbcc62c928c\") "
Sep 16 16:57:10.725981 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.725960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n49kd\" (UniqueName: \"kubernetes.io/projected/43d6cdf6-8400-4f62-a729-d8738fb88d06-kube-api-access-n49kd\") pod \"43d6cdf6-8400-4f62-a729-d8738fb88d06\" (UID: \"43d6cdf6-8400-4f62-a729-d8738fb88d06\") "
Sep 16 16:57:10.726319 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.726288    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d6cdf6-8400-4f62-a729-d8738fb88d06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "43d6cdf6-8400-4f62-a729-d8738fb88d06" (UID: "43d6cdf6-8400-4f62-a729-d8738fb88d06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:10.726319 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.726323    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1a6d3c8-3028-484f-adb6-efbcc62c928c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b1a6d3c8-3028-484f-adb6-efbcc62c928c" (UID: "b1a6d3c8-3028-484f-adb6-efbcc62c928c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:10.728157 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.728131    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1a6d3c8-3028-484f-adb6-efbcc62c928c-kube-api-access-d4lxh" (OuterVolumeSpecName: "kube-api-access-d4lxh") pod "b1a6d3c8-3028-484f-adb6-efbcc62c928c" (UID: "b1a6d3c8-3028-484f-adb6-efbcc62c928c"). InnerVolumeSpecName "kube-api-access-d4lxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:10.728239 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.728203    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d6cdf6-8400-4f62-a729-d8738fb88d06-kube-api-access-n49kd" (OuterVolumeSpecName: "kube-api-access-n49kd") pod "43d6cdf6-8400-4f62-a729-d8738fb88d06" (UID: "43d6cdf6-8400-4f62-a729-d8738fb88d06"). InnerVolumeSpecName "kube-api-access-n49kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:10.747295 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.747255    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"]
Sep 16 16:57:10.763084 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.763059    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-97b645-rsvks"]
Sep 16 16:57:10.802598 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.802546    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"]
Sep 16 16:57:10.805002 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:57:10.804978    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6468f446_19c1_4cb7_80fb_d6fae7971518.slice/crio-b42d82c2529263e0828fa28d30ad122eb63d3de9be0e3e85fb7e08206a21edcc WatchSource:0}: Error finding container b42d82c2529263e0828fa28d30ad122eb63d3de9be0e3e85fb7e08206a21edcc: Status 404 returned error can't find the container with id b42d82c2529263e0828fa28d30ad122eb63d3de9be0e3e85fb7e08206a21edcc
Sep 16 16:57:10.826846 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.826797    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43d6cdf6-8400-4f62-a729-d8738fb88d06-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:10.826846 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.826831    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4lxh\" (UniqueName: \"kubernetes.io/projected/b1a6d3c8-3028-484f-adb6-efbcc62c928c-kube-api-access-d4lxh\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:10.826846 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.826846    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1a6d3c8-3028-484f-adb6-efbcc62c928c-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:10.827070 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.826859    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n49kd\" (UniqueName: \"kubernetes.io/projected/43d6cdf6-8400-4f62-a729-d8738fb88d06-kube-api-access-n49kd\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:10.827070 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.826872    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:10.977593 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:10.977487    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa32618a-8a77-4461-b164-d645e8a0dba7" path="/var/lib/kubelet/pods/fa32618a-8a77-4461-b164-d645e8a0dba7/volumes"
Sep 16 16:57:11.088300 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.088267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:11.130933 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.130905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h" event={"ID":"6468f446-19c1-4cb7-80fb-d6fae7971518","Type":"ContainerStarted","Data":"57efb907b4dcc413bdf22a648fceb10c4c5fe64f4158d80cf6b84e3ab517fa96"}
Sep 16 16:57:11.131057 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.130958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h" event={"ID":"6468f446-19c1-4cb7-80fb-d6fae7971518","Type":"ContainerStarted","Data":"b42d82c2529263e0828fa28d30ad122eb63d3de9be0e3e85fb7e08206a21edcc"}
Sep 16 16:57:11.132207 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.132184    2577 generic.go:356] "Generic (PLEG): container finished" podID="b1a6d3c8-3028-484f-adb6-efbcc62c928c" containerID="a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d" exitCode=0
Sep 16 16:57:11.132312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.132218    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"
Sep 16 16:57:11.132312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.132261    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh" event={"ID":"b1a6d3c8-3028-484f-adb6-efbcc62c928c","Type":"ContainerDied","Data":"a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d"}
Sep 16 16:57:11.132312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.132305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh" event={"ID":"b1a6d3c8-3028-484f-adb6-efbcc62c928c","Type":"ContainerDied","Data":"87a37c57d82d437f4b059380e81583cf0e06f287b7ff7a75097f9b01e9d7c4d5"}
Sep 16 16:57:11.132490 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.132330    2577 scope.go:117] "RemoveContainer" containerID="a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d"
Sep 16 16:57:11.133692 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.133673    2577 generic.go:356] "Generic (PLEG): container finished" podID="43d6cdf6-8400-4f62-a729-d8738fb88d06" containerID="13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c" exitCode=0
Sep 16 16:57:11.133801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.133712    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"
Sep 16 16:57:11.133801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.133725    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs" event={"ID":"43d6cdf6-8400-4f62-a729-d8738fb88d06","Type":"ContainerDied","Data":"13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c"}
Sep 16 16:57:11.133801 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.133759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs" event={"ID":"43d6cdf6-8400-4f62-a729-d8738fb88d06","Type":"ContainerDied","Data":"53120a45f77475c893db9b73ad17738603eddcc6fa990ee17caa63ba5d93217f"}
Sep 16 16:57:11.141240 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.141222    2577 scope.go:117] "RemoveContainer" containerID="a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d"
Sep 16 16:57:11.141509 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:57:11.141491    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d\": container with ID starting with a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d not found: ID does not exist" containerID="a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d"
Sep 16 16:57:11.141583 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.141516    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d"} err="failed to get container status \"a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d\": rpc error: code = NotFound desc = could not find container \"a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d\": container with ID starting with a6d12eeb3cacd073b7645384ffd0608bcdef27f21c3ee85c9f3f5aaa3132886d not found: ID does not exist"
Sep 16 16:57:11.141583 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.141546    2577 scope.go:117] "RemoveContainer" containerID="13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c"
Sep 16 16:57:11.149245 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.149230    2577 scope.go:117] "RemoveContainer" containerID="13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c"
Sep 16 16:57:11.149504 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:57:11.149485    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c\": container with ID starting with 13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c not found: ID does not exist" containerID="13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c"
Sep 16 16:57:11.149504 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.149509    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c"} err="failed to get container status \"13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c\": rpc error: code = NotFound desc = could not find container \"13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c\": container with ID starting with 13858d03e8e0901b6040b7deb311ac51c0b3abfe9a16f5be60c73079724bdf2c not found: ID does not exist"
Sep 16 16:57:11.154017 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.153981    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h" podStartSLOduration=1.153969954 podStartE2EDuration="1.153969954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:11.152470666 +0000 UTC m=+1246.649343791" watchObservedRunningTime="2026-09-16 16:57:11.153969954 +0000 UTC m=+1246.650843078"
Sep 16 16:57:11.168872 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.168851    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"]
Sep 16 16:57:11.178167 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.178145    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-97b645-5696dc9dd9-xpdwh"]
Sep 16 16:57:11.191243 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.191219    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"]
Sep 16 16:57:11.196220 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.196196    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-97b645-9c77dbcdb-n5dbs"]
Sep 16 16:57:11.747194 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:11.747151    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:12.150444 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:12.150411    2577 generic.go:356] "Generic (PLEG): container finished" podID="6468f446-19c1-4cb7-80fb-d6fae7971518" containerID="57efb907b4dcc413bdf22a648fceb10c4c5fe64f4158d80cf6b84e3ab517fa96" exitCode=0
Sep 16 16:57:12.150645 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:12.150502    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h" event={"ID":"6468f446-19c1-4cb7-80fb-d6fae7971518","Type":"ContainerDied","Data":"57efb907b4dcc413bdf22a648fceb10c4c5fe64f4158d80cf6b84e3ab517fa96"}
Sep 16 16:57:12.763829 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:12.763793    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:12.977012 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:12.976977    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d6cdf6-8400-4f62-a729-d8738fb88d06" path="/var/lib/kubelet/pods/43d6cdf6-8400-4f62-a729-d8738fb88d06/volumes"
Sep 16 16:57:12.977325 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:12.977312    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1a6d3c8-3028-484f-adb6-efbcc62c928c" path="/var/lib/kubelet/pods/b1a6d3c8-3028-484f-adb6-efbcc62c928c/volumes"
Sep 16 16:57:13.290922 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:13.290897    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"
Sep 16 16:57:13.347976 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:13.347947    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6468f446-19c1-4cb7-80fb-d6fae7971518-maas-api-service-ca\") pod \"6468f446-19c1-4cb7-80fb-d6fae7971518\" (UID: \"6468f446-19c1-4cb7-80fb-d6fae7971518\") "
Sep 16 16:57:13.348283 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:13.348261    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6468f446-19c1-4cb7-80fb-d6fae7971518-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6468f446-19c1-4cb7-80fb-d6fae7971518" (UID: "6468f446-19c1-4cb7-80fb-d6fae7971518"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:13.449292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:13.449208    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6468f446-19c1-4cb7-80fb-d6fae7971518-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:13.575215 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:13.575184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:14.166094 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:14.166061    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h" event={"ID":"6468f446-19c1-4cb7-80fb-d6fae7971518","Type":"ContainerDied","Data":"b42d82c2529263e0828fa28d30ad122eb63d3de9be0e3e85fb7e08206a21edcc"}
Sep 16 16:57:14.166463 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:14.166114    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b42d82c2529263e0828fa28d30ad122eb63d3de9be0e3e85fb7e08206a21edcc"
Sep 16 16:57:14.166463 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:14.166067    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"
Sep 16 16:57:14.395852 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:14.395809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:15.049231 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.049203    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:15.080043 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.080013    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"]
Sep 16 16:57:15.080288 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.080256    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9" podUID="09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e" containerName="payload-processing" containerID="cri-o://14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9" gracePeriod=30
Sep 16 16:57:15.088862 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.088842    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"]
Sep 16 16:57:15.089042 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.089017    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd" podUID="3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0" containerName="payload-pre-processing" containerID="cri-o://62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026" gracePeriod=30
Sep 16 16:57:15.130496 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.130468    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:15.226056 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.226017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:15.324253 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.324222    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"]
Sep 16 16:57:15.329443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.329417    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f3a3973e-jphvm"]
Sep 16 16:57:15.338047 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.338024    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:57:15.361839 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.361820    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:57:15.381307 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.381282    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:15.467552 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.466803    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r2fx\" (UniqueName: \"kubernetes.io/projected/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-kube-api-access-5r2fx\") pod \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\" (UID: \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\") "
Sep 16 16:57:15.467552 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.466860    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4h4s\" (UniqueName: \"kubernetes.io/projected/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-kube-api-access-f4h4s\") pod \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\" (UID: \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\") "
Sep 16 16:57:15.467552 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.466937    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-plugins-config-volume\") pod \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\" (UID: \"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e\") "
Sep 16 16:57:15.467552 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.466991    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-plugins-config-volume\") pod \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\" (UID: \"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0\") "
Sep 16 16:57:15.467919 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.467698    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0" (UID: "3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:15.469929 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.468098    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e" (UID: "09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:15.471669 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.470556    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-kube-api-access-f4h4s" (OuterVolumeSpecName: "kube-api-access-f4h4s") pod "09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e" (UID: "09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e"). InnerVolumeSpecName "kube-api-access-f4h4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:15.476820 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.475620    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-kube-api-access-5r2fx" (OuterVolumeSpecName: "kube-api-access-5r2fx") pod "3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0" (UID: "3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0"). InnerVolumeSpecName "kube-api-access-5r2fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:15.568185 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.568114    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.568185 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.568141    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.568185 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.568150    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r2fx\" (UniqueName: \"kubernetes.io/projected/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0-kube-api-access-5r2fx\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.568185 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.568160    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4h4s\" (UniqueName: \"kubernetes.io/projected/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e-kube-api-access-f4h4s\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:15.665039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.665004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:15.889730 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:15.889693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:16.180023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.179932    2577 generic.go:356] "Generic (PLEG): container finished" podID="3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0" containerID="62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026" exitCode=0
Sep 16 16:57:16.180023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.179999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd" event={"ID":"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0","Type":"ContainerDied","Data":"62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026"}
Sep 16 16:57:16.180235 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.180056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd" event={"ID":"3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0","Type":"ContainerDied","Data":"446ee7866014a0e6ebf44b5065d1182a1d00787e15f331f8d7f7e8e263d07a5b"}
Sep 16 16:57:16.180235 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.180007    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"
Sep 16 16:57:16.180235 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.180083    2577 scope.go:117] "RemoveContainer" containerID="62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026"
Sep 16 16:57:16.181803 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.181781    2577 generic.go:356] "Generic (PLEG): container finished" podID="09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e" containerID="14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9" exitCode=0
Sep 16 16:57:16.181943 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.181852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9" event={"ID":"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e","Type":"ContainerDied","Data":"14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9"}
Sep 16 16:57:16.181943 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.181911    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9" event={"ID":"09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e","Type":"ContainerDied","Data":"b246e7901ef25d7973dda211b0f13537736499cbe63105f59627d02c8615ab68"}
Sep 16 16:57:16.181943 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.181856    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"
Sep 16 16:57:16.190573 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.190551    2577 scope.go:117] "RemoveContainer" containerID="62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026"
Sep 16 16:57:16.190856 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:57:16.190836    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026\": container with ID starting with 62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026 not found: ID does not exist" containerID="62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026"
Sep 16 16:57:16.190919 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.190864    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026"} err="failed to get container status \"62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026\": rpc error: code = NotFound desc = could not find container \"62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026\": container with ID starting with 62d6ae944cece4d6cdb0c9403d55b2e1d017af49c1c635bed4ad4d331085e026 not found: ID does not exist"
Sep 16 16:57:16.190919 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.190880    2577 scope.go:117] "RemoveContainer" containerID="14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9"
Sep 16 16:57:16.199567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.199550    2577 scope.go:117] "RemoveContainer" containerID="14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9"
Sep 16 16:57:16.199816 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:57:16.199799    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9\": container with ID starting with 14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9 not found: ID does not exist" containerID="14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9"
Sep 16 16:57:16.199901 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.199820    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9"} err="failed to get container status \"14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9\": rpc error: code = NotFound desc = could not find container \"14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9\": container with ID starting with 14b52fc9ce5e42d9556bc1446b8cf4d8a4de0ce86892a55e179e3961e4fc66f9 not found: ID does not exist"
Sep 16 16:57:16.209197 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.209175    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"]
Sep 16 16:57:16.215649 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.215628    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f3a3973e-68f9c64889-fxvz9"]
Sep 16 16:57:16.228639 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.228617    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"]
Sep 16 16:57:16.233382 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.233361    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f3a3973e-85c465665d-6hnxd"]
Sep 16 16:57:16.728765 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.728730    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:16.978914 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.978832    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e" path="/var/lib/kubelet/pods/09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e/volumes"
Sep 16 16:57:16.979229 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.979215    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0" path="/var/lib/kubelet/pods/3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0/volumes"
Sep 16 16:57:16.979504 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:16.979492    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7c698c4-8623-4765-8116-6fd42e0cc888" path="/var/lib/kubelet/pods/b7c698c4-8623-4765-8116-6fd42e0cc888/volumes"
Sep 16 16:57:17.667443 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:17.667407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:18.010562 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:18.010434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:18.119366 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:18.119334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:18.284396 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:18.284311    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:18.838593 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:18.838551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:19.325107 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:19.325062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:19.540939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:19.540907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:20.975931 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:20.975890    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:21.051288 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.051255    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"]
Sep 16 16:57:21.051572 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.051516    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs" podUID="e42abc1d-e075-4181-aaf2-90ab21812e0c" containerName="payload-processing" containerID="cri-o://341a093b856353bd27b5b5d2b03604795f44d1347cb7ecc665854cb60b78a78a" gracePeriod=30
Sep 16 16:57:21.084388 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.084359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:21.197513 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.197444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:21.222115 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.222084    2577 generic.go:356] "Generic (PLEG): container finished" podID="e42abc1d-e075-4181-aaf2-90ab21812e0c" containerID="341a093b856353bd27b5b5d2b03604795f44d1347cb7ecc665854cb60b78a78a" exitCode=0
Sep 16 16:57:21.222337 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.222309    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs" event={"ID":"e42abc1d-e075-4181-aaf2-90ab21812e0c","Type":"ContainerDied","Data":"341a093b856353bd27b5b5d2b03604795f44d1347cb7ecc665854cb60b78a78a"}
Sep 16 16:57:21.270441 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.270408    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"]
Sep 16 16:57:21.274431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.274403    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2ea3e-c476h"]
Sep 16 16:57:21.292200 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.292161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:21.322826 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.322804    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:57:21.426607 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.426570    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42abc1d-e075-4181-aaf2-90ab21812e0c-plugins-config-volume\") pod \"e42abc1d-e075-4181-aaf2-90ab21812e0c\" (UID: \"e42abc1d-e075-4181-aaf2-90ab21812e0c\") "
Sep 16 16:57:21.426772 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.426662    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vlv6\" (UniqueName: \"kubernetes.io/projected/e42abc1d-e075-4181-aaf2-90ab21812e0c-kube-api-access-5vlv6\") pod \"e42abc1d-e075-4181-aaf2-90ab21812e0c\" (UID: \"e42abc1d-e075-4181-aaf2-90ab21812e0c\") "
Sep 16 16:57:21.427013 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.426993    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42abc1d-e075-4181-aaf2-90ab21812e0c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e42abc1d-e075-4181-aaf2-90ab21812e0c" (UID: "e42abc1d-e075-4181-aaf2-90ab21812e0c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:21.428687 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.428670    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e42abc1d-e075-4181-aaf2-90ab21812e0c-kube-api-access-5vlv6" (OuterVolumeSpecName: "kube-api-access-5vlv6") pod "e42abc1d-e075-4181-aaf2-90ab21812e0c" (UID: "e42abc1d-e075-4181-aaf2-90ab21812e0c"). InnerVolumeSpecName "kube-api-access-5vlv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:21.528446 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.528367    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42abc1d-e075-4181-aaf2-90ab21812e0c-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:21.528446 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.528395    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vlv6\" (UniqueName: \"kubernetes.io/projected/e42abc1d-e075-4181-aaf2-90ab21812e0c-kube-api-access-5vlv6\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:21.617045 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.617005    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:21.717312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:21.717275    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:22.229086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.229052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs" event={"ID":"e42abc1d-e075-4181-aaf2-90ab21812e0c","Type":"ContainerDied","Data":"9979fe1e745d4362801a704367a0f882f209932341fd0fb16f62f35c298d384c"}
Sep 16 16:57:22.229513 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.229096    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"
Sep 16 16:57:22.229513 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.229112    2577 scope.go:117] "RemoveContainer" containerID="341a093b856353bd27b5b5d2b03604795f44d1347cb7ecc665854cb60b78a78a"
Sep 16 16:57:22.255312 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.255282    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"]
Sep 16 16:57:22.262289 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.262262    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2ea3e-66d68644c-jnvfs"]
Sep 16 16:57:22.520436 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.520354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:22.684563 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.684477    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:22.785654 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.785570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:22.940295 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.940267    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:22.977705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.977678    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6468f446-19c1-4cb7-80fb-d6fae7971518" path="/var/lib/kubelet/pods/6468f446-19c1-4cb7-80fb-d6fae7971518/volumes"
Sep 16 16:57:22.978082 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:22.978068    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e42abc1d-e075-4181-aaf2-90ab21812e0c" path="/var/lib/kubelet/pods/e42abc1d-e075-4181-aaf2-90ab21812e0c/volumes"
Sep 16 16:57:23.306720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:23.306685    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:23.408241 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:23.408206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:23.506842 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:23.506807    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:28.895742 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:28.895703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:29.066658 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.066619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:29.255478 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.255391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:29.551396 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551307    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"]
Sep 16 16:57:29.551775 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551757    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1a6d3c8-3028-484f-adb6-efbcc62c928c" containerName="payload-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551778    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1a6d3c8-3028-484f-adb6-efbcc62c928c" containerName="payload-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551793    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e" containerName="payload-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551802    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e" containerName="payload-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551817    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e42abc1d-e075-4181-aaf2-90ab21812e0c" containerName="payload-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551824    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e42abc1d-e075-4181-aaf2-90ab21812e0c" containerName="payload-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551838    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d6cdf6-8400-4f62-a729-d8738fb88d06" containerName="payload-pre-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551847    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d6cdf6-8400-4f62-a729-d8738fb88d06" containerName="payload-pre-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551860    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0" containerName="payload-pre-processing"
Sep 16 16:57:29.551880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551869    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0" containerName="payload-pre-processing"
Sep 16 16:57:29.552318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551899    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6468f446-19c1-4cb7-80fb-d6fae7971518" containerName="revoke-keys"
Sep 16 16:57:29.552318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.551908    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6468f446-19c1-4cb7-80fb-d6fae7971518" containerName="revoke-keys"
Sep 16 16:57:29.552318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.552037    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1a6d3c8-3028-484f-adb6-efbcc62c928c" containerName="payload-processing"
Sep 16 16:57:29.552318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.552047    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09dd0d1b-0bdf-454a-aa6f-b9e0b9c6c00e" containerName="payload-processing"
Sep 16 16:57:29.552318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.552058    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d6cdf6-8400-4f62-a729-d8738fb88d06" containerName="payload-pre-processing"
Sep 16 16:57:29.552318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.552068    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e42abc1d-e075-4181-aaf2-90ab21812e0c" containerName="payload-processing"
Sep 16 16:57:29.552318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.552080    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d2e815f-78bb-4ab1-9cfe-3d1a2134f8c0" containerName="payload-pre-processing"
Sep 16 16:57:29.552318 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.552090    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6468f446-19c1-4cb7-80fb-d6fae7971518" containerName="revoke-keys"
Sep 16 16:57:29.556679 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.556659    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"
Sep 16 16:57:29.559581 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.559562    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 16:57:29.559707 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.559630    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 16:57:29.570880 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.570859    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"]
Sep 16 16:57:29.695856 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.695816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b777647-8f3b-41a1-bde6-f2c5b281d59d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4\" (UID: \"9b777647-8f3b-41a1-bde6-f2c5b281d59d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"
Sep 16 16:57:29.797100 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.797063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b777647-8f3b-41a1-bde6-f2c5b281d59d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4\" (UID: \"9b777647-8f3b-41a1-bde6-f2c5b281d59d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"
Sep 16 16:57:29.797702 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.797682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b777647-8f3b-41a1-bde6-f2c5b281d59d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4\" (UID: \"9b777647-8f3b-41a1-bde6-f2c5b281d59d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"
Sep 16 16:57:29.866804 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.866743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"
Sep 16 16:57:29.876397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:29.876367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:30.152613 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:30.152585    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"]
Sep 16 16:57:30.153004 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:57:30.152679    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b777647_8f3b_41a1_bde6_f2c5b281d59d.slice/crio-dc45e8d7190d82faa17bcadba37807472073c46f39a5ce411261aedc71becf28 WatchSource:0}: Error finding container dc45e8d7190d82faa17bcadba37807472073c46f39a5ce411261aedc71becf28: Status 404 returned error can't find the container with id dc45e8d7190d82faa17bcadba37807472073c46f39a5ce411261aedc71becf28
Sep 16 16:57:30.276830 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:30.276797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4" event={"ID":"9b777647-8f3b-41a1-bde6-f2c5b281d59d","Type":"ContainerStarted","Data":"dc45e8d7190d82faa17bcadba37807472073c46f39a5ce411261aedc71becf28"}
Sep 16 16:57:30.901747 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:30.901715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:31.151814 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:31.151785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:31.259080 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:31.259050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:31.284586 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:31.284555    2577 generic.go:356] "Generic (PLEG): container finished" podID="9b777647-8f3b-41a1-bde6-f2c5b281d59d" containerID="8c01af949fb7ee27475209a9b3f6a09065ba27a2d65dcc018f1ca0abccf75e3e" exitCode=0
Sep 16 16:57:31.284754 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:31.284641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4" event={"ID":"9b777647-8f3b-41a1-bde6-f2c5b281d59d","Type":"ContainerDied","Data":"8c01af949fb7ee27475209a9b3f6a09065ba27a2d65dcc018f1ca0abccf75e3e"}
Sep 16 16:57:31.366987 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:31.366960    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:31.563442 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:31.563341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:31.973794 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:31.973759    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:32.424883 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:32.424857    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"
Sep 16 16:57:32.522054 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:32.522027    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b777647-8f3b-41a1-bde6-f2c5b281d59d-maas-api-service-ca\") pod \"9b777647-8f3b-41a1-bde6-f2c5b281d59d\" (UID: \"9b777647-8f3b-41a1-bde6-f2c5b281d59d\") "
Sep 16 16:57:32.522375 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:32.522354    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b777647-8f3b-41a1-bde6-f2c5b281d59d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b777647-8f3b-41a1-bde6-f2c5b281d59d" (UID: "9b777647-8f3b-41a1-bde6-f2c5b281d59d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:32.629819 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:32.624414    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b777647-8f3b-41a1-bde6-f2c5b281d59d-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:33.298142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:33.298057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4" event={"ID":"9b777647-8f3b-41a1-bde6-f2c5b281d59d","Type":"ContainerDied","Data":"dc45e8d7190d82faa17bcadba37807472073c46f39a5ce411261aedc71becf28"}
Sep 16 16:57:33.298142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:33.298108    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc45e8d7190d82faa17bcadba37807472073c46f39a5ce411261aedc71becf28"
Sep 16 16:57:33.298142 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:33.298067    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"
Sep 16 16:57:37.267905 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:37.267859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:37.367228 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:37.367181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:37.722688 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:37.722653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:38.128237 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:38.128151    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:38.426637 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:38.426559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:38.641547 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:38.641504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:40.483104 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:40.483067    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:41.092484 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.092444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:41.191960 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.191921    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:41.494793 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.494762    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:41.529500 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.529469    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"]
Sep 16 16:57:41.529829 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.529794    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w" podUID="34ad4da7-4a7f-432e-82ae-e9f4a995231e" containerName="payload-pre-processing" containerID="cri-o://51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271" gracePeriod=30
Sep 16 16:57:41.598714 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.598677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:41.696715 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.696515    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:41.747032 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.746944    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"]
Sep 16 16:57:41.750023 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.749992    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ea8f09-x5gn4"]
Sep 16 16:57:41.769199 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.769168    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:41.780152 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.780128    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:57:41.908614 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.908577    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad4da7-4a7f-432e-82ae-e9f4a995231e-plugins-config-volume\") pod \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\" (UID: \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\") "
Sep 16 16:57:41.908794 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.908694    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szsgt\" (UniqueName: \"kubernetes.io/projected/34ad4da7-4a7f-432e-82ae-e9f4a995231e-kube-api-access-szsgt\") pod \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\" (UID: \"34ad4da7-4a7f-432e-82ae-e9f4a995231e\") "
Sep 16 16:57:41.908968 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.908943    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34ad4da7-4a7f-432e-82ae-e9f4a995231e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34ad4da7-4a7f-432e-82ae-e9f4a995231e" (UID: "34ad4da7-4a7f-432e-82ae-e9f4a995231e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 16:57:41.910808 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:41.910785    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34ad4da7-4a7f-432e-82ae-e9f4a995231e-kube-api-access-szsgt" (OuterVolumeSpecName: "kube-api-access-szsgt") pod "34ad4da7-4a7f-432e-82ae-e9f4a995231e" (UID: "34ad4da7-4a7f-432e-82ae-e9f4a995231e"). InnerVolumeSpecName "kube-api-access-szsgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:57:42.009898 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.009807    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34ad4da7-4a7f-432e-82ae-e9f4a995231e-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:42.009898 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.009840    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szsgt\" (UniqueName: \"kubernetes.io/projected/34ad4da7-4a7f-432e-82ae-e9f4a995231e-kube-api-access-szsgt\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:57:42.104558 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.104498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:42.350556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.350447    2577 generic.go:356] "Generic (PLEG): container finished" podID="34ad4da7-4a7f-432e-82ae-e9f4a995231e" containerID="51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271" exitCode=0
Sep 16 16:57:42.350556 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.350512    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"
Sep 16 16:57:42.350770 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.350552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w" event={"ID":"34ad4da7-4a7f-432e-82ae-e9f4a995231e","Type":"ContainerDied","Data":"51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271"}
Sep 16 16:57:42.350770 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.350614    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w" event={"ID":"34ad4da7-4a7f-432e-82ae-e9f4a995231e","Type":"ContainerDied","Data":"8b975263e97713058b285eb53378c8fcfd7662c40c480bf0c78e4c96e2f3377a"}
Sep 16 16:57:42.350770 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.350673    2577 scope.go:117] "RemoveContainer" containerID="51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271"
Sep 16 16:57:42.360311 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.360291    2577 scope.go:117] "RemoveContainer" containerID="51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271"
Sep 16 16:57:42.360594 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:57:42.360572    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271\": container with ID starting with 51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271 not found: ID does not exist" containerID="51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271"
Sep 16 16:57:42.360662 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.360607    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271"} err="failed to get container status \"51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271\": rpc error: code = NotFound desc = could not find container \"51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271\": container with ID starting with 51be0a854e5ed3544efa47878adabbe7188238c29f5892723b9fd1d7b9fb9271 not found: ID does not exist"
Sep 16 16:57:42.375305 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.375283    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"]
Sep 16 16:57:42.382321 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.382299    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ea8f09-78fcf89fbb-x4r4w"]
Sep 16 16:57:42.521809 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.521770    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:42.611660 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.611578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:42.977397 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.977364    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34ad4da7-4a7f-432e-82ae-e9f4a995231e" path="/var/lib/kubelet/pods/34ad4da7-4a7f-432e-82ae-e9f4a995231e/volumes"
Sep 16 16:57:42.977752 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:42.977736    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b777647-8f3b-41a1-bde6-f2c5b281d59d" path="/var/lib/kubelet/pods/9b777647-8f3b-41a1-bde6-f2c5b281d59d/volumes"
Sep 16 16:57:43.074920 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.074884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:43.173972 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.173939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:43.194143 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.194103    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"]
Sep 16 16:57:43.194478 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.194466    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34ad4da7-4a7f-432e-82ae-e9f4a995231e" containerName="payload-pre-processing"
Sep 16 16:57:43.194562 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.194479    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="34ad4da7-4a7f-432e-82ae-e9f4a995231e" containerName="payload-pre-processing"
Sep 16 16:57:43.194562 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.194500    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b777647-8f3b-41a1-bde6-f2c5b281d59d" containerName="revoke-keys"
Sep 16 16:57:43.194562 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.194508    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b777647-8f3b-41a1-bde6-f2c5b281d59d" containerName="revoke-keys"
Sep 16 16:57:43.194679 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.194586    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="34ad4da7-4a7f-432e-82ae-e9f4a995231e" containerName="payload-pre-processing"
Sep 16 16:57:43.194679 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.194598    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b777647-8f3b-41a1-bde6-f2c5b281d59d" containerName="revoke-keys"
Sep 16 16:57:43.198815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.198799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:43.201686 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.201663    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-f315c3\""
Sep 16 16:57:43.201806 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.201713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-f315c3-dockercfg-6gvwp\""
Sep 16 16:57:43.212253 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.212229    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"]
Sep 16 16:57:43.218688 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.218662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p9fn\" (UniqueName: \"kubernetes.io/projected/2b1c872a-b985-4fde-9027-e342a5b8aa90-kube-api-access-9p9fn\") pod \"payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb\" (UID: \"2b1c872a-b985-4fde-9027-e342a5b8aa90\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:43.218783 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.218723    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1c872a-b985-4fde-9027-e342a5b8aa90-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb\" (UID: \"2b1c872a-b985-4fde-9027-e342a5b8aa90\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:43.269758 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.269693    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"]
Sep 16 16:57:43.273022 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.273007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:43.285138 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.285113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"]
Sep 16 16:57:43.319370 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.319341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p9fn\" (UniqueName: \"kubernetes.io/projected/2b1c872a-b985-4fde-9027-e342a5b8aa90-kube-api-access-9p9fn\") pod \"payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb\" (UID: \"2b1c872a-b985-4fde-9027-e342a5b8aa90\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:43.319514 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.319388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1c872a-b985-4fde-9027-e342a5b8aa90-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb\" (UID: \"2b1c872a-b985-4fde-9027-e342a5b8aa90\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:43.319514 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.319469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbf1b1d9-287c-41c8-9375-f2c666c00637-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn\" (UID: \"dbf1b1d9-287c-41c8-9375-f2c666c00637\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:43.319616 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.319594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t45d\" (UniqueName: \"kubernetes.io/projected/dbf1b1d9-287c-41c8-9375-f2c666c00637-kube-api-access-9t45d\") pod \"payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn\" (UID: \"dbf1b1d9-287c-41c8-9375-f2c666c00637\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:43.320144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.320128    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1c872a-b985-4fde-9027-e342a5b8aa90-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb\" (UID: \"2b1c872a-b985-4fde-9027-e342a5b8aa90\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:43.331251 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.331228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p9fn\" (UniqueName: \"kubernetes.io/projected/2b1c872a-b985-4fde-9027-e342a5b8aa90-kube-api-access-9p9fn\") pod \"payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb\" (UID: \"2b1c872a-b985-4fde-9027-e342a5b8aa90\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:43.421079 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.421046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbf1b1d9-287c-41c8-9375-f2c666c00637-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn\" (UID: \"dbf1b1d9-287c-41c8-9375-f2c666c00637\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:43.421274 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.421089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t45d\" (UniqueName: \"kubernetes.io/projected/dbf1b1d9-287c-41c8-9375-f2c666c00637-kube-api-access-9t45d\") pod \"payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn\" (UID: \"dbf1b1d9-287c-41c8-9375-f2c666c00637\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:43.421773 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.421750    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbf1b1d9-287c-41c8-9375-f2c666c00637-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn\" (UID: \"dbf1b1d9-287c-41c8-9375-f2c666c00637\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:43.429635 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.429612    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t45d\" (UniqueName: \"kubernetes.io/projected/dbf1b1d9-287c-41c8-9375-f2c666c00637-kube-api-access-9t45d\") pod \"payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn\" (UID: \"dbf1b1d9-287c-41c8-9375-f2c666c00637\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:43.509505 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.509465    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:43.583519 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.583486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:43.607303 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.607255    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:43.782466 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.782439    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"]
Sep 16 16:57:43.784091 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:57:43.784067    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b1c872a_b985_4fde_9027_e342a5b8aa90.slice/crio-c6aa269cf2806be19c1a0614bbbf2755f9081d05b797bcd95e4077b52a0d025d WatchSource:0}: Error finding container c6aa269cf2806be19c1a0614bbbf2755f9081d05b797bcd95e4077b52a0d025d: Status 404 returned error can't find the container with id c6aa269cf2806be19c1a0614bbbf2755f9081d05b797bcd95e4077b52a0d025d
Sep 16 16:57:43.873039 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:43.872093    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"]
Sep 16 16:57:44.366031 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.365997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb" event={"ID":"2b1c872a-b985-4fde-9027-e342a5b8aa90","Type":"ContainerStarted","Data":"0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581"}
Sep 16 16:57:44.366222 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.366054    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb" event={"ID":"2b1c872a-b985-4fde-9027-e342a5b8aa90","Type":"ContainerStarted","Data":"c6aa269cf2806be19c1a0614bbbf2755f9081d05b797bcd95e4077b52a0d025d"}
Sep 16 16:57:44.366294 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.366219    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:44.367963 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.367938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn" event={"ID":"dbf1b1d9-287c-41c8-9375-f2c666c00637","Type":"ContainerStarted","Data":"9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483"}
Sep 16 16:57:44.368051 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.367984    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn" event={"ID":"dbf1b1d9-287c-41c8-9375-f2c666c00637","Type":"ContainerStarted","Data":"dd84bbe4414c0c88939162cd97d54b845e5bf8e06ec94c77f3598f86c65072bf"}
Sep 16 16:57:44.368114 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.368060    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:44.384167 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.384118    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb" podStartSLOduration=1.384104714 podStartE2EDuration="1.384104714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:44.38308208 +0000 UTC m=+1279.879955206" watchObservedRunningTime="2026-09-16 16:57:44.384104714 +0000 UTC m=+1279.880977878"
Sep 16 16:57:44.401939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.401895    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn" podStartSLOduration=1.401882417 podStartE2EDuration="1.401882417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:44.399492133 +0000 UTC m=+1279.896365261" watchObservedRunningTime="2026-09-16 16:57:44.401882417 +0000 UTC m=+1279.898755541"
Sep 16 16:57:44.444833 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:44.444803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:48.440625 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:48.440588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:48.545359 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:48.545321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:48.709806 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:48.709716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:49.226295 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:49.226252    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:49.641762 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:49.641722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:49.959679 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:49.959584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:50.697946 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:50.697911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:50.807377 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:50.807343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:51.011689 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:51.011603    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:51.111767 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:51.111725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:52.484353 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:52.484319    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:53.167292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:53.167238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:53.251944 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:53.251902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:53.353198 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:53.353155    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:54.399333 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:54.399294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:54.478358 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:54.478321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:54.612286 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:54.612248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:55.028451 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.028413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:55.122430 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.122390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:55.135645 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.135619    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"]
Sep 16 16:57:55.138913 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.138893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:55.140793 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.140773    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"]
Sep 16 16:57:55.141576 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.141548    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-52fd8c\""
Sep 16 16:57:55.143927 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.143912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:55.157727 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.157706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"]
Sep 16 16:57:55.173215 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.173188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"]
Sep 16 16:57:55.224661 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.224635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd\" (UID: \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:55.224796 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.224697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klswx\" (UniqueName: \"kubernetes.io/projected/bc7e98c8-31fe-45be-8383-716bc44e5f9f-kube-api-access-klswx\") pod \"payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b\" (UID: \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:55.224869 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.224801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc7e98c8-31fe-45be-8383-716bc44e5f9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b\" (UID: \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:55.224869 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.224840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg79w\" (UniqueName: \"kubernetes.io/projected/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-kube-api-access-xg79w\") pod \"payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd\" (UID: \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:55.325661 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.325574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klswx\" (UniqueName: \"kubernetes.io/projected/bc7e98c8-31fe-45be-8383-716bc44e5f9f-kube-api-access-klswx\") pod \"payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b\" (UID: \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:55.325661 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.325639    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc7e98c8-31fe-45be-8383-716bc44e5f9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b\" (UID: \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:55.325901 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.325676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg79w\" (UniqueName: \"kubernetes.io/projected/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-kube-api-access-xg79w\") pod \"payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd\" (UID: \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:55.325901 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.325698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd\" (UID: \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:55.326497 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.326472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd\" (UID: \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:55.326596 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.326480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc7e98c8-31fe-45be-8383-716bc44e5f9f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b\" (UID: \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:55.338113 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.338086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klswx\" (UniqueName: \"kubernetes.io/projected/bc7e98c8-31fe-45be-8383-716bc44e5f9f-kube-api-access-klswx\") pod \"payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b\" (UID: \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:55.338254 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.338229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg79w\" (UniqueName: \"kubernetes.io/projected/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-kube-api-access-xg79w\") pod \"payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd\" (UID: \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:55.375405 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.375373    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 16:57:55.375592 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.375574    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 16:57:55.450351 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.450320    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:55.457159 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.457134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:55.521128 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.520911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:55.752072 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.752040    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"]
Sep 16 16:57:55.753676 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:57:55.753651    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5896cf57_8f2f_4f1f_9d51_bd2fdc00231a.slice/crio-c42d6aa92f39855498a899bf14b61f4fd75d62781137e5a112a2b68c953c6532 WatchSource:0}: Error finding container c42d6aa92f39855498a899bf14b61f4fd75d62781137e5a112a2b68c953c6532: Status 404 returned error can't find the container with id c42d6aa92f39855498a899bf14b61f4fd75d62781137e5a112a2b68c953c6532
Sep 16 16:57:55.756260 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.756239    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"]
Sep 16 16:57:55.757630 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:57:55.757458    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc7e98c8_31fe_45be_8383_716bc44e5f9f.slice/crio-c4747818b220c331aae2c8ed47f4de81215278f1fb86983463076ef6be37e3f6 WatchSource:0}: Error finding container c4747818b220c331aae2c8ed47f4de81215278f1fb86983463076ef6be37e3f6: Status 404 returned error can't find the container with id c4747818b220c331aae2c8ed47f4de81215278f1fb86983463076ef6be37e3f6
Sep 16 16:57:55.804593 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:55.804551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:56.076610 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.076522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:56.175720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.175682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:56.368130 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.368047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:56.445257 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.445222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd" event={"ID":"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a","Type":"ContainerStarted","Data":"0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be"}
Sep 16 16:57:56.445447 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.445285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd" event={"ID":"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a","Type":"ContainerStarted","Data":"c42d6aa92f39855498a899bf14b61f4fd75d62781137e5a112a2b68c953c6532"}
Sep 16 16:57:56.445447 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.445332    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:57:56.446932 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.446906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b" event={"ID":"bc7e98c8-31fe-45be-8383-716bc44e5f9f","Type":"ContainerStarted","Data":"0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6"}
Sep 16 16:57:56.447087 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.446954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b" event={"ID":"bc7e98c8-31fe-45be-8383-716bc44e5f9f","Type":"ContainerStarted","Data":"c4747818b220c331aae2c8ed47f4de81215278f1fb86983463076ef6be37e3f6"}
Sep 16 16:57:56.447087 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.447004    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:57:56.466321 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.466274    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd" podStartSLOduration=1.466260828 podStartE2EDuration="1.466260828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:56.464717685 +0000 UTC m=+1291.961590819" watchObservedRunningTime="2026-09-16 16:57:56.466260828 +0000 UTC m=+1291.963133951"
Sep 16 16:57:56.489716 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.489676    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b" podStartSLOduration=1.489665241 podStartE2EDuration="1.489665241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:57:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:57:56.48697559 +0000 UTC m=+1291.983848715" watchObservedRunningTime="2026-09-16 16:57:56.489665241 +0000 UTC m=+1291.986538420"
Sep 16 16:57:56.492623 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.492597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:56.666336 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:56.666298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:57.229970 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:57.229933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:59.305308 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:59.305273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:57:59.914979 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:57:59.914943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:00.011549 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:00.011493    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:00.387373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:00.387334    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:01.218771 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:01.218729    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:02.310680 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:02.310646    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:02.409746 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:02.409712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:02.516564 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:02.516523    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:02.943436 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:02.943401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:03.025431 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.025393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:03.063584 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.063558    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"]
Sep 16 16:58:03.067182 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.067167    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:03.073739 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.073714    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-2ce8bc\""
Sep 16 16:58:03.073739 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.073733    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-2ce8bc-dockercfg-j82zk\""
Sep 16 16:58:03.079646 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.079624    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"]
Sep 16 16:58:03.195189 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.195098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d64df8-9242-40d0-b9bb-0ca5229d4fed-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd\" (UID: \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:03.195361 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.195243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s9ds\" (UniqueName: \"kubernetes.io/projected/95d64df8-9242-40d0-b9bb-0ca5229d4fed-kube-api-access-7s9ds\") pod \"payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd\" (UID: \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:03.295939 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.295903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d64df8-9242-40d0-b9bb-0ca5229d4fed-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd\" (UID: \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:03.296103 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.296003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s9ds\" (UniqueName: \"kubernetes.io/projected/95d64df8-9242-40d0-b9bb-0ca5229d4fed-kube-api-access-7s9ds\") pod \"payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd\" (UID: \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:03.296610 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.296591    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d64df8-9242-40d0-b9bb-0ca5229d4fed-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd\" (UID: \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:03.305027 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.305003    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s9ds\" (UniqueName: \"kubernetes.io/projected/95d64df8-9242-40d0-b9bb-0ca5229d4fed-kube-api-access-7s9ds\") pod \"payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd\" (UID: \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:03.335006 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.334979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:03.378681 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.378654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:03.651572 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:03.651541    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"]
Sep 16 16:58:03.652773 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:58:03.652748    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d64df8_9242_40d0_b9bb_0ca5229d4fed.slice/crio-2062a6d16d24b5e8c5a7fa2b4fcf7f06a13c43f62b9322aeaa65dc7b0c033c49 WatchSource:0}: Error finding container 2062a6d16d24b5e8c5a7fa2b4fcf7f06a13c43f62b9322aeaa65dc7b0c033c49: Status 404 returned error can't find the container with id 2062a6d16d24b5e8c5a7fa2b4fcf7f06a13c43f62b9322aeaa65dc7b0c033c49
Sep 16 16:58:04.267682 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:04.267636    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:04.498064 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:04.498025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd" event={"ID":"95d64df8-9242-40d0-b9bb-0ca5229d4fed","Type":"ContainerStarted","Data":"ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38"}
Sep 16 16:58:04.498462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:04.498078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd" event={"ID":"95d64df8-9242-40d0-b9bb-0ca5229d4fed","Type":"ContainerStarted","Data":"2062a6d16d24b5e8c5a7fa2b4fcf7f06a13c43f62b9322aeaa65dc7b0c033c49"}
Sep 16 16:58:04.498462 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:04.498103    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:04.517436 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:04.517389    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd" podStartSLOduration=1.517370925 podStartE2EDuration="1.517370925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:04.514627962 +0000 UTC m=+1300.011501108" watchObservedRunningTime="2026-09-16 16:58:04.517370925 +0000 UTC m=+1300.014244049"
Sep 16 16:58:04.993904 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:04.993866    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:05.772166 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:05.772129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:06.594937 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:06.594897    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:07.354458 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:07.354423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:07.454423 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:07.454395    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 16:58:07.454598 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:07.454461    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 16:58:07.824320 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:07.824288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:08.137926 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:08.137890    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:08.239539 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:08.239502    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:08.456550 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:08.456440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:08.553144 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:08.553114    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:11.392780 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:11.392737    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:11.901912 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:11.901869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:11.998340 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:11.998292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:12.135286 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:12.135239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:13.594779 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:13.594737    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:14.355945 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:14.355903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:15.219690 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.219655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:15.326588 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.326520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:15.505673 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.505579    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 16:58:15.583279 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.583243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:15.757179 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.757077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:15.849894 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.849851    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:15.856732 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.856703    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"]
Sep 16 16:58:15.860502 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.860486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:15.863699 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.863673    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-34d301\""
Sep 16 16:58:15.879473 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:15.879449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"]
Sep 16 16:58:16.015248 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.015171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d1c623a-888e-404e-9192-520d547daa22-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k\" (UID: \"5d1c623a-888e-404e-9192-520d547daa22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:16.015248 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.015239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n8qc\" (UniqueName: \"kubernetes.io/projected/5d1c623a-888e-404e-9192-520d547daa22-kube-api-access-2n8qc\") pod \"payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k\" (UID: \"5d1c623a-888e-404e-9192-520d547daa22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:16.116766 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.116735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d1c623a-888e-404e-9192-520d547daa22-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k\" (UID: \"5d1c623a-888e-404e-9192-520d547daa22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:16.116925 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.116805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n8qc\" (UniqueName: \"kubernetes.io/projected/5d1c623a-888e-404e-9192-520d547daa22-kube-api-access-2n8qc\") pod \"payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k\" (UID: \"5d1c623a-888e-404e-9192-520d547daa22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:16.117468 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.117434    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d1c623a-888e-404e-9192-520d547daa22-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k\" (UID: \"5d1c623a-888e-404e-9192-520d547daa22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:16.132100 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.132070    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n8qc\" (UniqueName: \"kubernetes.io/projected/5d1c623a-888e-404e-9192-520d547daa22-kube-api-access-2n8qc\") pod \"payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k\" (UID: \"5d1c623a-888e-404e-9192-520d547daa22\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:16.171187 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.171160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:16.202835 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.202800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:16.451034 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.451005    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"]
Sep 16 16:58:16.452522 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:58:16.452498    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1c623a_888e_404e_9192_520d547daa22.slice/crio-e4ddc25d379b07f55168cf0ec79f41125ededed3a17692a1ae71b0e01b909479 WatchSource:0}: Error finding container e4ddc25d379b07f55168cf0ec79f41125ededed3a17692a1ae71b0e01b909479: Status 404 returned error can't find the container with id e4ddc25d379b07f55168cf0ec79f41125ededed3a17692a1ae71b0e01b909479
Sep 16 16:58:16.572141 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.572106    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k" event={"ID":"5d1c623a-888e-404e-9192-520d547daa22","Type":"ContainerStarted","Data":"a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790"}
Sep 16 16:58:16.572334 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.572170    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:16.572334 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.572189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k" event={"ID":"5d1c623a-888e-404e-9192-520d547daa22","Type":"ContainerStarted","Data":"e4ddc25d379b07f55168cf0ec79f41125ededed3a17692a1ae71b0e01b909479"}
Sep 16 16:58:16.593121 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:16.593067    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k" podStartSLOduration=1.5930484470000001 podStartE2EDuration="1.593048447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:16.592298287 +0000 UTC m=+1312.089171421" watchObservedRunningTime="2026-09-16 16:58:16.593048447 +0000 UTC m=+1312.089921573"
Sep 16 16:58:17.273230 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:17.273177    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:18.283751 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:18.283709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:19.208711 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:19.208678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:19.670858 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:19.670815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:20.036215 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.036112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:20.435143 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.435109    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7917-552867"]
Sep 16 16:58:20.439191 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.439169    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-552867"
Sep 16 16:58:20.442168 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.442149    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rcrsh\""
Sep 16 16:58:20.449853 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.449832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-552867"]
Sep 16 16:58:20.558604 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.558566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfjs4\" (UniqueName: \"kubernetes.io/projected/42330500-1bd9-4975-b0ac-79f1edd719bf-kube-api-access-nfjs4\") pod \"test-curl-7917-552867\" (UID: \"42330500-1bd9-4975-b0ac-79f1edd719bf\") " pod="opendatahub/test-curl-7917-552867"
Sep 16 16:58:20.659444 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.659413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfjs4\" (UniqueName: \"kubernetes.io/projected/42330500-1bd9-4975-b0ac-79f1edd719bf-kube-api-access-nfjs4\") pod \"test-curl-7917-552867\" (UID: \"42330500-1bd9-4975-b0ac-79f1edd719bf\") " pod="opendatahub/test-curl-7917-552867"
Sep 16 16:58:20.668993 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.668963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfjs4\" (UniqueName: \"kubernetes.io/projected/42330500-1bd9-4975-b0ac-79f1edd719bf-kube-api-access-nfjs4\") pod \"test-curl-7917-552867\" (UID: \"42330500-1bd9-4975-b0ac-79f1edd719bf\") " pod="opendatahub/test-curl-7917-552867"
Sep 16 16:58:20.749050 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:20.748984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-552867"
Sep 16 16:58:21.028938 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:21.028906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-552867"]
Sep 16 16:58:21.029081 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:58:21.029019    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42330500_1bd9_4975_b0ac_79f1edd719bf.slice/crio-c29e8d8d5d23cc0cffc5db838fcb5f19b46631999ff1d66720ea699dc07870e9 WatchSource:0}: Error finding container c29e8d8d5d23cc0cffc5db838fcb5f19b46631999ff1d66720ea699dc07870e9: Status 404 returned error can't find the container with id c29e8d8d5d23cc0cffc5db838fcb5f19b46631999ff1d66720ea699dc07870e9
Sep 16 16:58:21.608096 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:21.608060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-552867" event={"ID":"42330500-1bd9-4975-b0ac-79f1edd719bf","Type":"ContainerStarted","Data":"187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1"}
Sep 16 16:58:21.608285 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:21.608109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-552867" event={"ID":"42330500-1bd9-4975-b0ac-79f1edd719bf","Type":"ContainerStarted","Data":"c29e8d8d5d23cc0cffc5db838fcb5f19b46631999ff1d66720ea699dc07870e9"}
Sep 16 16:58:21.625578 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:21.625512    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7917-552867" podStartSLOduration=1.625498845 podStartE2EDuration="1.625498845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:21.624223865 +0000 UTC m=+1317.121096992" watchObservedRunningTime="2026-09-16 16:58:21.625498845 +0000 UTC m=+1317.122371968"
Sep 16 16:58:22.230695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.230663    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7917-552867"]
Sep 16 16:58:22.234178 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.234146    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7917-552867"]
Sep 16 16:58:22.346750 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.346674    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7917-4af692"]
Sep 16 16:58:22.347086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.347072    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42330500-1bd9-4975-b0ac-79f1edd719bf" containerName="test-curl-7917-552867"
Sep 16 16:58:22.347134 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.347087    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="42330500-1bd9-4975-b0ac-79f1edd719bf" containerName="test-curl-7917-552867"
Sep 16 16:58:22.347170 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.347158    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="42330500-1bd9-4975-b0ac-79f1edd719bf" containerName="test-curl-7917-552867"
Sep 16 16:58:22.350284 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.350266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-4af692"
Sep 16 16:58:22.354743 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.354719    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-4af692"]
Sep 16 16:58:22.477341 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.477311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhtws\" (UniqueName: \"kubernetes.io/projected/39b4355d-bd97-4588-9f4c-7e6871a2921f-kube-api-access-bhtws\") pod \"test-curl-7917-4af692\" (UID: \"39b4355d-bd97-4588-9f4c-7e6871a2921f\") " pod="opendatahub/test-curl-7917-4af692"
Sep 16 16:58:22.578265 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.578230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhtws\" (UniqueName: \"kubernetes.io/projected/39b4355d-bd97-4588-9f4c-7e6871a2921f-kube-api-access-bhtws\") pod \"test-curl-7917-4af692\" (UID: \"39b4355d-bd97-4588-9f4c-7e6871a2921f\") " pod="opendatahub/test-curl-7917-4af692"
Sep 16 16:58:22.586471 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.586445    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhtws\" (UniqueName: \"kubernetes.io/projected/39b4355d-bd97-4588-9f4c-7e6871a2921f-kube-api-access-bhtws\") pod \"test-curl-7917-4af692\" (UID: \"39b4355d-bd97-4588-9f4c-7e6871a2921f\") " pod="opendatahub/test-curl-7917-4af692"
Sep 16 16:58:22.614087 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:22.614026    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7917-552867" container="test-curl-7917-552867"
Sep 16 16:58:22.660498 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.660470    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-4af692"
Sep 16 16:58:22.931665 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:22.931635    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-4af692"]
Sep 16 16:58:22.932389 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:58:22.932364    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39b4355d_bd97_4588_9f4c_7e6871a2921f.slice/crio-414957d88608e5e1e4b7d8fc8b432c1d02c57a1f9f4b2289ff501febc3578bed WatchSource:0}: Error finding container 414957d88608e5e1e4b7d8fc8b432c1d02c57a1f9f4b2289ff501febc3578bed: Status 404 returned error can't find the container with id 414957d88608e5e1e4b7d8fc8b432c1d02c57a1f9f4b2289ff501febc3578bed
Sep 16 16:58:23.454928 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:23.454895    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:23.621780 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:23.621752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-4af692" event={"ID":"39b4355d-bd97-4588-9f4c-7e6871a2921f","Type":"ContainerStarted","Data":"55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087"}
Sep 16 16:58:23.621780 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:23.621789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-4af692" event={"ID":"39b4355d-bd97-4588-9f4c-7e6871a2921f","Type":"ContainerStarted","Data":"414957d88608e5e1e4b7d8fc8b432c1d02c57a1f9f4b2289ff501febc3578bed"}
Sep 16 16:58:23.622043 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:23.621869    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7917-552867" podUID="42330500-1bd9-4975-b0ac-79f1edd719bf" containerName="test-curl-7917-552867" containerID="cri-o://187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1" gracePeriod=2
Sep 16 16:58:23.640392 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:23.640350    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7917-4af692" podStartSLOduration=1.6403351480000001 podStartE2EDuration="1.640335148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:23.637727885 +0000 UTC m=+1319.134601021" watchObservedRunningTime="2026-09-16 16:58:23.640335148 +0000 UTC m=+1319.137208272"
Sep 16 16:58:24.258899 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.258863    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7917-4af692"]
Sep 16 16:58:24.263267 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.263227    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7917-4af692"]
Sep 16 16:58:24.376674 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.376594    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7917-c5778c"]
Sep 16 16:58:24.377003 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.376990    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b4355d-bd97-4588-9f4c-7e6871a2921f" containerName="test-curl-7917-4af692"
Sep 16 16:58:24.377061 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.377004    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b4355d-bd97-4588-9f4c-7e6871a2921f" containerName="test-curl-7917-4af692"
Sep 16 16:58:24.377098 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.377073    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b4355d-bd97-4588-9f4c-7e6871a2921f" containerName="test-curl-7917-4af692"
Sep 16 16:58:24.380108 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.380091    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-c5778c"
Sep 16 16:58:24.386889 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.386861    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-c5778c"]
Sep 16 16:58:24.496757 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.496728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t2hr\" (UniqueName: \"kubernetes.io/projected/1d335683-7851-44e1-9057-b7e2c854869d-kube-api-access-9t2hr\") pod \"test-curl-7917-c5778c\" (UID: \"1d335683-7851-44e1-9057-b7e2c854869d\") " pod="opendatahub/test-curl-7917-c5778c"
Sep 16 16:58:24.597575 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.597522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t2hr\" (UniqueName: \"kubernetes.io/projected/1d335683-7851-44e1-9057-b7e2c854869d-kube-api-access-9t2hr\") pod \"test-curl-7917-c5778c\" (UID: \"1d335683-7851-44e1-9057-b7e2c854869d\") " pod="opendatahub/test-curl-7917-c5778c"
Sep 16 16:58:24.607171 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.607148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t2hr\" (UniqueName: \"kubernetes.io/projected/1d335683-7851-44e1-9057-b7e2c854869d-kube-api-access-9t2hr\") pod \"test-curl-7917-c5778c\" (UID: \"1d335683-7851-44e1-9057-b7e2c854869d\") " pod="opendatahub/test-curl-7917-c5778c"
Sep 16 16:58:24.628269 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:24.628210    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7917-4af692" container="test-curl-7917-4af692"
Sep 16 16:58:24.690152 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.690115    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-c5778c"
Sep 16 16:58:24.962051 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:24.962020    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-c5778c"]
Sep 16 16:58:24.964510 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:58:24.964482    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d335683_7851_44e1_9057_b7e2c854869d.slice/crio-792d7bbbca4f82f5878d6014555b3fbddc6ac4e8b41c97659a70c6f31edb7ee8 WatchSource:0}: Error finding container 792d7bbbca4f82f5878d6014555b3fbddc6ac4e8b41c97659a70c6f31edb7ee8: Status 404 returned error can't find the container with id 792d7bbbca4f82f5878d6014555b3fbddc6ac4e8b41c97659a70c6f31edb7ee8
Sep 16 16:58:25.636065 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:25.636031    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-c5778c" event={"ID":"1d335683-7851-44e1-9057-b7e2c854869d","Type":"ContainerStarted","Data":"5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15"}
Sep 16 16:58:25.636425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:25.636086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-c5778c" event={"ID":"1d335683-7851-44e1-9057-b7e2c854869d","Type":"ContainerStarted","Data":"792d7bbbca4f82f5878d6014555b3fbddc6ac4e8b41c97659a70c6f31edb7ee8"}
Sep 16 16:58:25.636425 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:25.636233    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7917-4af692" podUID="39b4355d-bd97-4588-9f4c-7e6871a2921f" containerName="test-curl-7917-4af692" containerID="cri-o://55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087" gracePeriod=2
Sep 16 16:58:25.652627 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:25.652581    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7917-c5778c" podStartSLOduration=1.652566792 podStartE2EDuration="1.652566792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:25.651522512 +0000 UTC m=+1321.148395637" watchObservedRunningTime="2026-09-16 16:58:25.652566792 +0000 UTC m=+1321.149439916"
Sep 16 16:58:25.794361 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:25.794327    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-552867"
Sep 16 16:58:25.908453 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:25.908425    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfjs4\" (UniqueName: \"kubernetes.io/projected/42330500-1bd9-4975-b0ac-79f1edd719bf-kube-api-access-nfjs4\") pod \"42330500-1bd9-4975-b0ac-79f1edd719bf\" (UID: \"42330500-1bd9-4975-b0ac-79f1edd719bf\") "
Sep 16 16:58:25.910292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:25.910263    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42330500-1bd9-4975-b0ac-79f1edd719bf-kube-api-access-nfjs4" (OuterVolumeSpecName: "kube-api-access-nfjs4") pod "42330500-1bd9-4975-b0ac-79f1edd719bf" (UID: "42330500-1bd9-4975-b0ac-79f1edd719bf"). InnerVolumeSpecName "kube-api-access-nfjs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:58:26.009968 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.009927    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfjs4\" (UniqueName: \"kubernetes.io/projected/42330500-1bd9-4975-b0ac-79f1edd719bf-kube-api-access-nfjs4\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:58:26.223596 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.223502    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7917-c5778c"]
Sep 16 16:58:26.228483 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.228457    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7917-c5778c"]
Sep 16 16:58:26.340369 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.340335    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7917-1c3fab"]
Sep 16 16:58:26.340775 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.340759    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d335683-7851-44e1-9057-b7e2c854869d" containerName="test-curl-7917-c5778c"
Sep 16 16:58:26.340830 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.340777    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d335683-7851-44e1-9057-b7e2c854869d" containerName="test-curl-7917-c5778c"
Sep 16 16:58:26.340866 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.340848    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d335683-7851-44e1-9057-b7e2c854869d" containerName="test-curl-7917-c5778c"
Sep 16 16:58:26.343911 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.343894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-1c3fab"
Sep 16 16:58:26.348796 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.348772    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-1c3fab"]
Sep 16 16:58:26.513588 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.513493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb49j\" (UniqueName: \"kubernetes.io/projected/a38f57e6-0266-42dc-a292-d9de5c64f61f-kube-api-access-nb49j\") pod \"test-curl-7917-1c3fab\" (UID: \"a38f57e6-0266-42dc-a292-d9de5c64f61f\") " pod="opendatahub/test-curl-7917-1c3fab"
Sep 16 16:58:26.614838 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.614805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb49j\" (UniqueName: \"kubernetes.io/projected/a38f57e6-0266-42dc-a292-d9de5c64f61f-kube-api-access-nb49j\") pod \"test-curl-7917-1c3fab\" (UID: \"a38f57e6-0266-42dc-a292-d9de5c64f61f\") " pod="opendatahub/test-curl-7917-1c3fab"
Sep 16 16:58:26.623742 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.623708    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb49j\" (UniqueName: \"kubernetes.io/projected/a38f57e6-0266-42dc-a292-d9de5c64f61f-kube-api-access-nb49j\") pod \"test-curl-7917-1c3fab\" (UID: \"a38f57e6-0266-42dc-a292-d9de5c64f61f\") " pod="opendatahub/test-curl-7917-1c3fab"
Sep 16 16:58:26.643287 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.643257    2577 generic.go:356] "Generic (PLEG): container finished" podID="42330500-1bd9-4975-b0ac-79f1edd719bf" containerID="187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1" exitCode=137
Sep 16 16:58:26.643653 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.643302    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-552867"
Sep 16 16:58:26.643653 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.643347    2577 scope.go:117] "RemoveContainer" containerID="187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1"
Sep 16 16:58:26.643761 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:26.643717    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7917-c5778c" container="test-curl-7917-c5778c"
Sep 16 16:58:26.653154 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.653138    2577 scope.go:117] "RemoveContainer" containerID="187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1"
Sep 16 16:58:26.653425 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:26.653405    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1\": container with ID starting with 187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1 not found: ID does not exist" containerID="187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1"
Sep 16 16:58:26.653496 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.653434    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1"} err="failed to get container status \"187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1\": rpc error: code = NotFound desc = could not find container \"187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1\": container with ID starting with 187379c952cd14633691202f506a4c23981abb362a1a863d496cba6a0d3b96b1 not found: ID does not exist"
Sep 16 16:58:26.654189 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.654178    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-1c3fab"
Sep 16 16:58:26.654461 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:26.654440    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-552867\" is forbidden: User \"system:node:ip-10-0-139-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-51.ec2.internal' and this object" podUID="42330500-1bd9-4975-b0ac-79f1edd719bf" pod="opendatahub/test-curl-7917-552867"
Sep 16 16:58:26.923874 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.923844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7917-1c3fab"]
Sep 16 16:58:26.925819 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:58:26.925777    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda38f57e6_0266_42dc_a292_d9de5c64f61f.slice/crio-64ab9e4e9cce71bb50ded39cee11e543e28af8a2a085d7f4af4d2d2035a72184 WatchSource:0}: Error finding container 64ab9e4e9cce71bb50ded39cee11e543e28af8a2a085d7f4af4d2d2035a72184: Status 404 returned error can't find the container with id 64ab9e4e9cce71bb50ded39cee11e543e28af8a2a085d7f4af4d2d2035a72184
Sep 16 16:58:26.978828 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:26.978801    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42330500-1bd9-4975-b0ac-79f1edd719bf" path="/var/lib/kubelet/pods/42330500-1bd9-4975-b0ac-79f1edd719bf/volumes"
Sep 16 16:58:27.580782 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.580751    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 16:58:27.654373 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.654339    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-1c3fab" event={"ID":"a38f57e6-0266-42dc-a292-d9de5c64f61f","Type":"ContainerStarted","Data":"98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2"}
Sep 16 16:58:27.654785 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.654390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7917-1c3fab" event={"ID":"a38f57e6-0266-42dc-a292-d9de5c64f61f","Type":"ContainerStarted","Data":"64ab9e4e9cce71bb50ded39cee11e543e28af8a2a085d7f4af4d2d2035a72184"}
Sep 16 16:58:27.655445 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.655421    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7917-c5778c" podUID="1d335683-7851-44e1-9057-b7e2c854869d" containerName="test-curl-7917-c5778c" containerID="cri-o://5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15" gracePeriod=2
Sep 16 16:58:27.689517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.689470    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7917-1c3fab" podStartSLOduration=1.68945222 podStartE2EDuration="1.68945222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:27.686794607 +0000 UTC m=+1323.183667732" watchObservedRunningTime="2026-09-16 16:58:27.68945222 +0000 UTC m=+1323.186325343"
Sep 16 16:58:27.825718 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.825695    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-4af692"
Sep 16 16:58:27.826057 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.826040    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhtws\" (UniqueName: \"kubernetes.io/projected/39b4355d-bd97-4588-9f4c-7e6871a2921f-kube-api-access-bhtws\") pod \"39b4355d-bd97-4588-9f4c-7e6871a2921f\" (UID: \"39b4355d-bd97-4588-9f4c-7e6871a2921f\") "
Sep 16 16:58:27.828086 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.828066    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b4355d-bd97-4588-9f4c-7e6871a2921f-kube-api-access-bhtws" (OuterVolumeSpecName: "kube-api-access-bhtws") pod "39b4355d-bd97-4588-9f4c-7e6871a2921f" (UID: "39b4355d-bd97-4588-9f4c-7e6871a2921f"). InnerVolumeSpecName "kube-api-access-bhtws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:58:27.927126 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:27.927098    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhtws\" (UniqueName: \"kubernetes.io/projected/39b4355d-bd97-4588-9f4c-7e6871a2921f-kube-api-access-bhtws\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:58:28.288232 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.288152    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7917-1c3fab"]
Sep 16 16:58:28.291101 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.291072    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7917-1c3fab"]
Sep 16 16:58:28.663205 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.663166    2577 generic.go:356] "Generic (PLEG): container finished" podID="39b4355d-bd97-4588-9f4c-7e6871a2921f" containerID="55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087" exitCode=137
Sep 16 16:58:28.663619 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.663228    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-4af692"
Sep 16 16:58:28.663619 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.663229    2577 scope.go:117] "RemoveContainer" containerID="55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087"
Sep 16 16:58:28.663619 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:28.663613    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7917-1c3fab" container="test-curl-7917-1c3fab"
Sep 16 16:58:28.673235 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.673217    2577 scope.go:117] "RemoveContainer" containerID="55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087"
Sep 16 16:58:28.673500 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:28.673480    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087\": container with ID starting with 55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087 not found: ID does not exist" containerID="55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087"
Sep 16 16:58:28.673591 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.673506    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087"} err="failed to get container status \"55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087\": rpc error: code = NotFound desc = could not find container \"55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087\": container with ID starting with 55b525dd1595a632866d20f2e96e8cbaf7e86b55ac75da95a44feb88623ae087 not found: ID does not exist"
Sep 16 16:58:28.673707 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:28.673690    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-4af692\" is forbidden: User \"system:node:ip-10-0-139-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-51.ec2.internal' and this object" podUID="39b4355d-bd97-4588-9f4c-7e6871a2921f" pod="opendatahub/test-curl-7917-4af692"
Sep 16 16:58:28.875572 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.875517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:28.978210 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:28.978132    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b4355d-bd97-4588-9f4c-7e6871a2921f" path="/var/lib/kubelet/pods/39b4355d-bd97-4588-9f4c-7e6871a2921f/volumes"
Sep 16 16:58:29.195308 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:29.195273    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:29.470484 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:29.470452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:29.670762 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:29.670728    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7917-1c3fab" podUID="a38f57e6-0266-42dc-a292-d9de5c64f61f" containerName="test-curl-7917-1c3fab" containerID="cri-o://98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2" gracePeriod=2
Sep 16 16:58:29.825097 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:29.825072    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-c5778c"
Sep 16 16:58:29.842247 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:29.842216    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t2hr\" (UniqueName: \"kubernetes.io/projected/1d335683-7851-44e1-9057-b7e2c854869d-kube-api-access-9t2hr\") pod \"1d335683-7851-44e1-9057-b7e2c854869d\" (UID: \"1d335683-7851-44e1-9057-b7e2c854869d\") "
Sep 16 16:58:29.844241 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:29.844216    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d335683-7851-44e1-9057-b7e2c854869d-kube-api-access-9t2hr" (OuterVolumeSpecName: "kube-api-access-9t2hr") pod "1d335683-7851-44e1-9057-b7e2c854869d" (UID: "1d335683-7851-44e1-9057-b7e2c854869d"). InnerVolumeSpecName "kube-api-access-9t2hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:58:29.943759 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:29.943727    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t2hr\" (UniqueName: \"kubernetes.io/projected/1d335683-7851-44e1-9057-b7e2c854869d-kube-api-access-9t2hr\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:58:30.679137 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:30.679102    2577 generic.go:356] "Generic (PLEG): container finished" podID="1d335683-7851-44e1-9057-b7e2c854869d" containerID="5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15" exitCode=137
Sep 16 16:58:30.679600 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:30.679153    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-c5778c"
Sep 16 16:58:30.679600 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:30.679206    2577 scope.go:117] "RemoveContainer" containerID="5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15"
Sep 16 16:58:30.688399 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:30.688382    2577 scope.go:117] "RemoveContainer" containerID="5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15"
Sep 16 16:58:30.688683 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:30.688665    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15\": container with ID starting with 5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15 not found: ID does not exist" containerID="5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15"
Sep 16 16:58:30.688747 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:30.688692    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15"} err="failed to get container status \"5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15\": rpc error: code = NotFound desc = could not find container \"5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15\": container with ID starting with 5688176d68b76e392038923920c2a4db0a119f9e250613aa15ad214110a1cb15 not found: ID does not exist"
Sep 16 16:58:30.690368 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:30.690346    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-c5778c\" is forbidden: User \"system:node:ip-10-0-139-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-51.ec2.internal' and this object" podUID="1d335683-7851-44e1-9057-b7e2c854869d" pod="opendatahub/test-curl-7917-c5778c"
Sep 16 16:58:30.977991 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:30.977915    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d335683-7851-44e1-9057-b7e2c854869d" path="/var/lib/kubelet/pods/1d335683-7851-44e1-9057-b7e2c854869d/volumes"
Sep 16 16:58:31.469846 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:31.469815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:31.859727 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:31.859703    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-1c3fab"
Sep 16 16:58:31.963249 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:31.963220    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb49j\" (UniqueName: \"kubernetes.io/projected/a38f57e6-0266-42dc-a292-d9de5c64f61f-kube-api-access-nb49j\") pod \"a38f57e6-0266-42dc-a292-d9de5c64f61f\" (UID: \"a38f57e6-0266-42dc-a292-d9de5c64f61f\") "
Sep 16 16:58:31.965252 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:31.965231    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a38f57e6-0266-42dc-a292-d9de5c64f61f-kube-api-access-nb49j" (OuterVolumeSpecName: "kube-api-access-nb49j") pod "a38f57e6-0266-42dc-a292-d9de5c64f61f" (UID: "a38f57e6-0266-42dc-a292-d9de5c64f61f"). InnerVolumeSpecName "kube-api-access-nb49j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 16:58:32.064946 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.064913    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb49j\" (UniqueName: \"kubernetes.io/projected/a38f57e6-0266-42dc-a292-d9de5c64f61f-kube-api-access-nb49j\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 16:58:32.320504 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.320408    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:32.458721 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.458679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:32.566567 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.566509    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:32.695695 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.695657    2577 generic.go:356] "Generic (PLEG): container finished" podID="a38f57e6-0266-42dc-a292-d9de5c64f61f" containerID="98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2" exitCode=137
Sep 16 16:58:32.695889 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.695702    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7917-1c3fab"
Sep 16 16:58:32.695889 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.695759    2577 scope.go:117] "RemoveContainer" containerID="98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2"
Sep 16 16:58:32.705239 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.705222    2577 scope.go:117] "RemoveContainer" containerID="98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2"
Sep 16 16:58:32.705499 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:32.705472    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2\": container with ID starting with 98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2 not found: ID does not exist" containerID="98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2"
Sep 16 16:58:32.705626 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.705507    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2"} err="failed to get container status \"98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2\": rpc error: code = NotFound desc = could not find container \"98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2\": container with ID starting with 98cf5bc7fc0f3c0a3a2d0be4d89497e8bc53d99a03bcd3d3162aff777f1cb6e2 not found: ID does not exist"
Sep 16 16:58:32.705690 ip-10-0-139-51 kubenswrapper[2577]: E0916 16:58:32.705673    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7917-1c3fab\" is forbidden: User \"system:node:ip-10-0-139-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-51.ec2.internal' and this object" podUID="a38f57e6-0266-42dc-a292-d9de5c64f61f" pod="opendatahub/test-curl-7917-1c3fab"
Sep 16 16:58:32.978605 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:32.978490    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a38f57e6-0266-42dc-a292-d9de5c64f61f" path="/var/lib/kubelet/pods/a38f57e6-0266-42dc-a292-d9de5c64f61f/volumes"
Sep 16 16:58:33.520183 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:33.520141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:34.281001 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:34.280966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:34.384411 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:34.384374    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:35.301705 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:35.301665    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:36.261881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.261851    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:36.590289 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.590202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:36.781641 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.781607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:36.881805 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.881716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:36.900077 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.900045    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"]
Sep 16 16:58:36.900464 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.900452    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a38f57e6-0266-42dc-a292-d9de5c64f61f" containerName="test-curl-7917-1c3fab"
Sep 16 16:58:36.900517 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.900465    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a38f57e6-0266-42dc-a292-d9de5c64f61f" containerName="test-curl-7917-1c3fab"
Sep 16 16:58:36.900572 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.900547    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a38f57e6-0266-42dc-a292-d9de5c64f61f" containerName="test-curl-7917-1c3fab"
Sep 16 16:58:36.904815 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.904800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:36.907481 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.907461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6148d487-dockercfg-q9mt6\""
Sep 16 16:58:36.907720 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.907706    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6148d487\""
Sep 16 16:58:36.918493 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:36.918471    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"]
Sep 16 16:58:37.012885 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.012845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgtbv\" (UniqueName: \"kubernetes.io/projected/9cb67476-44eb-41fd-a62d-740ff3df45b8-kube-api-access-zgtbv\") pod \"payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2\" (UID: \"9cb67476-44eb-41fd-a62d-740ff3df45b8\") " pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:37.013088 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.013013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cb67476-44eb-41fd-a62d-740ff3df45b8-plugins-config-volume\") pod \"payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2\" (UID: \"9cb67476-44eb-41fd-a62d-740ff3df45b8\") " pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:37.114181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.114139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgtbv\" (UniqueName: \"kubernetes.io/projected/9cb67476-44eb-41fd-a62d-740ff3df45b8-kube-api-access-zgtbv\") pod \"payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2\" (UID: \"9cb67476-44eb-41fd-a62d-740ff3df45b8\") " pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:37.114363 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.114198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cb67476-44eb-41fd-a62d-740ff3df45b8-plugins-config-volume\") pod \"payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2\" (UID: \"9cb67476-44eb-41fd-a62d-740ff3df45b8\") " pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:37.114934 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.114914    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cb67476-44eb-41fd-a62d-740ff3df45b8-plugins-config-volume\") pod \"payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2\" (UID: \"9cb67476-44eb-41fd-a62d-740ff3df45b8\") " pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:37.131254 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.131226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgtbv\" (UniqueName: \"kubernetes.io/projected/9cb67476-44eb-41fd-a62d-740ff3df45b8-kube-api-access-zgtbv\") pod \"payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2\" (UID: \"9cb67476-44eb-41fd-a62d-740ff3df45b8\") " pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:37.215136 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.215079    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:37.244033 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.243983    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:37.348119 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.348088    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:37.502367 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.502342    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"]
Sep 16 16:58:37.504003 ip-10-0-139-51 kubenswrapper[2577]: W0916 16:58:37.503969    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb67476_44eb_41fd_a62d_740ff3df45b8.slice/crio-797cf5790e9115a90961a5069e4b0df845e97c264a7587a495b2cebdad638983 WatchSource:0}: Error finding container 797cf5790e9115a90961a5069e4b0df845e97c264a7587a495b2cebdad638983: Status 404 returned error can't find the container with id 797cf5790e9115a90961a5069e4b0df845e97c264a7587a495b2cebdad638983
Sep 16 16:58:37.728762 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.728721    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2" event={"ID":"9cb67476-44eb-41fd-a62d-740ff3df45b8","Type":"ContainerStarted","Data":"406be075d888a7a20bcb33d8dc3ed1bfbc5f7724c7ae2e2c06b36d9ef18e1736"}
Sep 16 16:58:37.729213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.728778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2" event={"ID":"9cb67476-44eb-41fd-a62d-740ff3df45b8","Type":"ContainerStarted","Data":"797cf5790e9115a90961a5069e4b0df845e97c264a7587a495b2cebdad638983"}
Sep 16 16:58:37.729213 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.728865    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:37.754524 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:37.754438    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2" podStartSLOduration=1.754421399 podStartE2EDuration="1.754421399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 16:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 16:58:37.751936161 +0000 UTC m=+1333.248809286" watchObservedRunningTime="2026-09-16 16:58:37.754421399 +0000 UTC m=+1333.251294528"
Sep 16 16:58:38.061146 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:38.061036    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:38.929785 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:38.929744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:39.792696 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:39.792659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:40.519786 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:40.519750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:41.336728 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:41.336678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:45.560698 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:45.560664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:48.739292 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:48.739223    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 16:58:58.465870 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:58.465828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:59.170181 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:59.170133    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:58:59.395724 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:58:59.395684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:59:00.118038 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:00.117997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:59:00.810743 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:00.810698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:59:01.609490 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:01.609456    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:59:02.172454 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:02.172419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:59:02.384984 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:02.384953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:59:12.978218 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:12.978178    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 16:59:25.116881 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:25.116850    2577 scope.go:117] "RemoveContainer" containerID="771f7fb59c51d234d24460bd1b4485f8db6ebd0b739f117bc205bbabb069c90f"
Sep 16 16:59:25.125562 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:25.125544    2577 scope.go:117] "RemoveContainer" containerID="c3dee5e682bf6ebcd0d1a9fdee3bc016253416e617cad2897f60e0f127056be2"
Sep 16 16:59:42.485185 ip-10-0-139-51 kubenswrapper[2577]: I0916 16:59:42.485145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:00.144590 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.144555    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"]
Sep 16 17:00:00.148552 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.148517    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"
Sep 16 17:00:00.148852 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.148829    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"]
Sep 16 17:00:00.151054 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.151036    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:00:00.152099 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.152082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"
Sep 16 17:00:00.152307 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.152293    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"]
Sep 16 17:00:00.155388 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.155372    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"
Sep 16 17:00:00.155757 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.155731    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"]
Sep 16 17:00:00.159615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.159592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"
Sep 16 17:00:00.159824 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.159801    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"]
Sep 16 17:00:00.168511 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.168484    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"]
Sep 16 17:00:00.169825 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.169625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"
Sep 16 17:00:00.170621 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.170487    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"]
Sep 16 17:00:00.175642 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.175619    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"]
Sep 16 17:00:00.182016 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.181985    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"
Sep 16 17:00:00.234672 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.234644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"
Sep 16 17:00:00.245804 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.245610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"
Sep 16 17:00:00.462261 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.462235    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"]
Sep 16 17:00:00.463768 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:00:00.463740    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01248bb0_512c_4adc_9497_c6a8b1f43f07.slice/crio-dfa4d4689f476fc6e20dd86e64d886b5de39e6ec5ee74dd060fa9b57a2af6738 WatchSource:0}: Error finding container dfa4d4689f476fc6e20dd86e64d886b5de39e6ec5ee74dd060fa9b57a2af6738: Status 404 returned error can't find the container with id dfa4d4689f476fc6e20dd86e64d886b5de39e6ec5ee74dd060fa9b57a2af6738
Sep 16 17:00:00.477381 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.477358    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"]
Sep 16 17:00:00.479149 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:00:00.479127    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ff2aa5_60ce_4f15_a55d_b0851d4cbeb3.slice/crio-a29b6e426984cd5da0a5ef5788cd468aff18eea29a3d1b24dac5dd2cd6d1b968 WatchSource:0}: Error finding container a29b6e426984cd5da0a5ef5788cd468aff18eea29a3d1b24dac5dd2cd6d1b968: Status 404 returned error can't find the container with id a29b6e426984cd5da0a5ef5788cd468aff18eea29a3d1b24dac5dd2cd6d1b968
Sep 16 17:00:00.534874 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.534843    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"]
Sep 16 17:00:00.536660 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:00:00.536636    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d16c34_9219_448d_95a5_1732d7d403a0.slice/crio-72c1bf6711df5ce544589dd8ecba144c10d11ef4eb52cd279de838c2ce21a66c WatchSource:0}: Error finding container 72c1bf6711df5ce544589dd8ecba144c10d11ef4eb52cd279de838c2ce21a66c: Status 404 returned error can't find the container with id 72c1bf6711df5ce544589dd8ecba144c10d11ef4eb52cd279de838c2ce21a66c
Sep 16 17:00:00.538636 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:00.538613    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"]
Sep 16 17:00:00.539439 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:00:00.539416    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9af77317_6884_4c84_a4a2_5b7a42195060.slice/crio-790419cd15647b3e69abfe6768e26bf22302ee9dacf1474f9da5923d4aeacfc4 WatchSource:0}: Error finding container 790419cd15647b3e69abfe6768e26bf22302ee9dacf1474f9da5923d4aeacfc4: Status 404 returned error can't find the container with id 790419cd15647b3e69abfe6768e26bf22302ee9dacf1474f9da5923d4aeacfc4
Sep 16 17:00:01.274170 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.274132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz" event={"ID":"01248bb0-512c-4adc-9497-c6a8b1f43f07","Type":"ContainerStarted","Data":"aae0da164fce5611b635cf5aede79b5a4a9025e2a1a57ed08181a7735869ae60"}
Sep 16 17:00:01.274628 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.274193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz" event={"ID":"01248bb0-512c-4adc-9497-c6a8b1f43f07","Type":"ContainerStarted","Data":"dfa4d4689f476fc6e20dd86e64d886b5de39e6ec5ee74dd060fa9b57a2af6738"}
Sep 16 17:00:01.275784 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.275759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk" event={"ID":"81d16c34-9219-448d-95a5-1732d7d403a0","Type":"ContainerStarted","Data":"f9aa889706913e1c1f4e7858e0b5f0b33d7a58858418a51996355cb218692b76"}
Sep 16 17:00:01.275919 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.275802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk" event={"ID":"81d16c34-9219-448d-95a5-1732d7d403a0","Type":"ContainerStarted","Data":"72c1bf6711df5ce544589dd8ecba144c10d11ef4eb52cd279de838c2ce21a66c"}
Sep 16 17:00:01.277272 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.277243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx" event={"ID":"9af77317-6884-4c84-a4a2-5b7a42195060","Type":"ContainerStarted","Data":"1501fe5a3a0a895416babcb066a5b79cfa8e414ad9fbb164b6dd9f23695aa668"}
Sep 16 17:00:01.277377 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.277293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx" event={"ID":"9af77317-6884-4c84-a4a2-5b7a42195060","Type":"ContainerStarted","Data":"790419cd15647b3e69abfe6768e26bf22302ee9dacf1474f9da5923d4aeacfc4"}
Sep 16 17:00:01.278923 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.278902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2" event={"ID":"13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3","Type":"ContainerStarted","Data":"842ba3405e8e1b2f0d6eb3311054e30651e8994836b5fb9bd2a102e7d2b69e6d"}
Sep 16 17:00:01.279009 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.278937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2" event={"ID":"13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3","Type":"ContainerStarted","Data":"a29b6e426984cd5da0a5ef5788cd468aff18eea29a3d1b24dac5dd2cd6d1b968"}
Sep 16 17:00:01.290502 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.290448    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz" podStartSLOduration=1.290430251 podStartE2EDuration="1.290430251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:00:01.288041125 +0000 UTC m=+1416.784914252" watchObservedRunningTime="2026-09-16 17:00:01.290430251 +0000 UTC m=+1416.787303376"
Sep 16 17:00:01.304313 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.304272    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk" podStartSLOduration=1.304263581 podStartE2EDuration="1.304263581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:00:01.301229956 +0000 UTC m=+1416.798103081" watchObservedRunningTime="2026-09-16 17:00:01.304263581 +0000 UTC m=+1416.801136772"
Sep 16 17:00:01.316719 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.316680    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx" podStartSLOduration=1.316668599 podStartE2EDuration="1.316668599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:00:01.31435381 +0000 UTC m=+1416.811226934" watchObservedRunningTime="2026-09-16 17:00:01.316668599 +0000 UTC m=+1416.813541745"
Sep 16 17:00:01.329521 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:01.329483    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2" podStartSLOduration=1.329472089 podStartE2EDuration="1.329472089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:00:01.328207617 +0000 UTC m=+1416.825080738" watchObservedRunningTime="2026-09-16 17:00:01.329472089 +0000 UTC m=+1416.826345215"
Sep 16 17:00:02.286440 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:02.286408    2577 generic.go:356] "Generic (PLEG): container finished" podID="81d16c34-9219-448d-95a5-1732d7d403a0" containerID="f9aa889706913e1c1f4e7858e0b5f0b33d7a58858418a51996355cb218692b76" exitCode=0
Sep 16 17:00:02.286865 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:02.286498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk" event={"ID":"81d16c34-9219-448d-95a5-1732d7d403a0","Type":"ContainerDied","Data":"f9aa889706913e1c1f4e7858e0b5f0b33d7a58858418a51996355cb218692b76"}
Sep 16 17:00:02.288209 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:02.288180    2577 generic.go:356] "Generic (PLEG): container finished" podID="9af77317-6884-4c84-a4a2-5b7a42195060" containerID="1501fe5a3a0a895416babcb066a5b79cfa8e414ad9fbb164b6dd9f23695aa668" exitCode=0
Sep 16 17:00:02.288333 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:02.288265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx" event={"ID":"9af77317-6884-4c84-a4a2-5b7a42195060","Type":"ContainerDied","Data":"1501fe5a3a0a895416babcb066a5b79cfa8e414ad9fbb164b6dd9f23695aa668"}
Sep 16 17:00:02.289848 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:02.289824    2577 generic.go:356] "Generic (PLEG): container finished" podID="13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3" containerID="842ba3405e8e1b2f0d6eb3311054e30651e8994836b5fb9bd2a102e7d2b69e6d" exitCode=0
Sep 16 17:00:02.289968 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:02.289901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2" event={"ID":"13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3","Type":"ContainerDied","Data":"842ba3405e8e1b2f0d6eb3311054e30651e8994836b5fb9bd2a102e7d2b69e6d"}
Sep 16 17:00:02.291442 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:02.291424    2577 generic.go:356] "Generic (PLEG): container finished" podID="01248bb0-512c-4adc-9497-c6a8b1f43f07" containerID="aae0da164fce5611b635cf5aede79b5a4a9025e2a1a57ed08181a7735869ae60" exitCode=0
Sep 16 17:00:02.291552 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:02.291480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz" event={"ID":"01248bb0-512c-4adc-9497-c6a8b1f43f07","Type":"ContainerDied","Data":"aae0da164fce5611b635cf5aede79b5a4a9025e2a1a57ed08181a7735869ae60"}
Sep 16 17:00:03.480598 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:03.480569    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"
Sep 16 17:00:03.484173 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:03.484152    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"
Sep 16 17:00:03.517969 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:03.517935    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"
Sep 16 17:00:03.521376 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:03.521354    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"
Sep 16 17:00:03.694641 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:03.694604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:04.306340 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.306310    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"
Sep 16 17:00:04.306522 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.306342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2" event={"ID":"13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3","Type":"ContainerDied","Data":"a29b6e426984cd5da0a5ef5788cd468aff18eea29a3d1b24dac5dd2cd6d1b968"}
Sep 16 17:00:04.306522 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.306387    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a29b6e426984cd5da0a5ef5788cd468aff18eea29a3d1b24dac5dd2cd6d1b968"
Sep 16 17:00:04.307859 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.307827    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"
Sep 16 17:00:04.307859 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.307841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz" event={"ID":"01248bb0-512c-4adc-9497-c6a8b1f43f07","Type":"ContainerDied","Data":"dfa4d4689f476fc6e20dd86e64d886b5de39e6ec5ee74dd060fa9b57a2af6738"}
Sep 16 17:00:04.308045 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.307888    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfa4d4689f476fc6e20dd86e64d886b5de39e6ec5ee74dd060fa9b57a2af6738"
Sep 16 17:00:04.309415 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.309387    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"
Sep 16 17:00:04.309503 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.309414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk" event={"ID":"81d16c34-9219-448d-95a5-1732d7d403a0","Type":"ContainerDied","Data":"72c1bf6711df5ce544589dd8ecba144c10d11ef4eb52cd279de838c2ce21a66c"}
Sep 16 17:00:04.309503 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.309465    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72c1bf6711df5ce544589dd8ecba144c10d11ef4eb52cd279de838c2ce21a66c"
Sep 16 17:00:04.310823 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.310802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx" event={"ID":"9af77317-6884-4c84-a4a2-5b7a42195060","Type":"ContainerDied","Data":"790419cd15647b3e69abfe6768e26bf22302ee9dacf1474f9da5923d4aeacfc4"}
Sep 16 17:00:04.310910 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.310826    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"
Sep 16 17:00:04.310910 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:04.310847    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="790419cd15647b3e69abfe6768e26bf22302ee9dacf1474f9da5923d4aeacfc4"
Sep 16 17:00:22.655395 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.655302    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"]
Sep 16 17:00:22.655984 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.655962    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01248bb0-512c-4adc-9497-c6a8b1f43f07" containerName="cleanup"
Sep 16 17:00:22.656066 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.655988    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="01248bb0-512c-4adc-9497-c6a8b1f43f07" containerName="cleanup"
Sep 16 17:00:22.656066 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656015    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9af77317-6884-4c84-a4a2-5b7a42195060" containerName="cleanup"
Sep 16 17:00:22.656066 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656025    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9af77317-6884-4c84-a4a2-5b7a42195060" containerName="cleanup"
Sep 16 17:00:22.656066 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656036    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3" containerName="cleanup"
Sep 16 17:00:22.656066 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656046    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3" containerName="cleanup"
Sep 16 17:00:22.656328 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656089    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81d16c34-9219-448d-95a5-1732d7d403a0" containerName="cleanup"
Sep 16 17:00:22.656328 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656098    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="81d16c34-9219-448d-95a5-1732d7d403a0" containerName="cleanup"
Sep 16 17:00:22.656328 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656192    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3" containerName="cleanup"
Sep 16 17:00:22.656328 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656205    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="01248bb0-512c-4adc-9497-c6a8b1f43f07" containerName="cleanup"
Sep 16 17:00:22.656328 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656219    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9af77317-6884-4c84-a4a2-5b7a42195060" containerName="cleanup"
Sep 16 17:00:22.656328 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.656230    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="81d16c34-9219-448d-95a5-1732d7d403a0" containerName="cleanup"
Sep 16 17:00:22.659863 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.659841    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"
Sep 16 17:00:22.662654 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.662627    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:00:22.662878 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.662848    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:00:22.665847 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.665825    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"]
Sep 16 17:00:22.787503 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.787463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68d17f-a353-4861-8660-702869e5777d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6148d487-trffd\" (UID: \"6e68d17f-a353-4861-8660-702869e5777d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"
Sep 16 17:00:22.888327 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.888288    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68d17f-a353-4861-8660-702869e5777d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6148d487-trffd\" (UID: \"6e68d17f-a353-4861-8660-702869e5777d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"
Sep 16 17:00:22.888947 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.888926    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68d17f-a353-4861-8660-702869e5777d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6148d487-trffd\" (UID: \"6e68d17f-a353-4861-8660-702869e5777d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"
Sep 16 17:00:22.956137 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.956046    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:22.973178 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:22.973146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"
Sep 16 17:00:23.250296 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:23.250268    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"]
Sep 16 17:00:23.251662 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:00:23.251640    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e68d17f_a353_4861_8660_702869e5777d.slice/crio-5a2bfd7c27ad565ac37d2c41e4cf0ccccfdf7e7c969bfd55f1b93598927c0772 WatchSource:0}: Error finding container 5a2bfd7c27ad565ac37d2c41e4cf0ccccfdf7e7c969bfd55f1b93598927c0772: Status 404 returned error can't find the container with id 5a2bfd7c27ad565ac37d2c41e4cf0ccccfdf7e7c969bfd55f1b93598927c0772
Sep 16 17:00:23.428652 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:23.428619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd" event={"ID":"6e68d17f-a353-4861-8660-702869e5777d","Type":"ContainerStarted","Data":"5a2bfd7c27ad565ac37d2c41e4cf0ccccfdf7e7c969bfd55f1b93598927c0772"}
Sep 16 17:00:23.811975 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:23.811880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:24.437522 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:24.437489    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e68d17f-a353-4861-8660-702869e5777d" containerID="b842bec0447b0c3a70c634fddc94e0d74966258ae39d76463434fa3b2b98f8e0" exitCode=0
Sep 16 17:00:24.437709 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:24.437570    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd" event={"ID":"6e68d17f-a353-4861-8660-702869e5777d","Type":"ContainerDied","Data":"b842bec0447b0c3a70c634fddc94e0d74966258ae39d76463434fa3b2b98f8e0"}
Sep 16 17:00:24.560847 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:24.560809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:25.166800 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:25.166761    2577 scope.go:117] "RemoveContainer" containerID="0ff3c728319ae4e2592210bd858af2d72f83fb433d113c09b5f40577a447367f"
Sep 16 17:00:25.440282 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:25.440181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:25.585048 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:25.585018    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"
Sep 16 17:00:25.718585 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:25.718473    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68d17f-a353-4861-8660-702869e5777d-maas-api-service-ca\") pod \"6e68d17f-a353-4861-8660-702869e5777d\" (UID: \"6e68d17f-a353-4861-8660-702869e5777d\") "
Sep 16 17:00:25.718879 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:25.718855    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e68d17f-a353-4861-8660-702869e5777d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e68d17f-a353-4861-8660-702869e5777d" (UID: "6e68d17f-a353-4861-8660-702869e5777d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:00:25.819414 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:25.819369    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e68d17f-a353-4861-8660-702869e5777d-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:00:26.389260 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:26.389222    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:26.453152 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:26.453121    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"
Sep 16 17:00:26.453310 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:26.453150    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd" event={"ID":"6e68d17f-a353-4861-8660-702869e5777d","Type":"ContainerDied","Data":"5a2bfd7c27ad565ac37d2c41e4cf0ccccfdf7e7c969bfd55f1b93598927c0772"}
Sep 16 17:00:26.453310 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:26.453194    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a2bfd7c27ad565ac37d2c41e4cf0ccccfdf7e7c969bfd55f1b93598927c0772"
Sep 16 17:00:27.076735 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:27.076700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:27.772207 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:27.772165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:28.923347 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:28.923306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:38.315846 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.315814    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"]
Sep 16 17:00:38.318443 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.318419    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-6148d487-29826300-596fz"]
Sep 16 17:00:38.341292 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.341259    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"]
Sep 16 17:00:38.341561 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.341496    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2" podUID="9cb67476-44eb-41fd-a62d-740ff3df45b8" containerName="payload-processing" containerID="cri-o://406be075d888a7a20bcb33d8dc3ed1bfbc5f7724c7ae2e2c06b36d9ef18e1736" gracePeriod=30
Sep 16 17:00:38.437727 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.437688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:38.533839 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.533810    2577 generic.go:356] "Generic (PLEG): container finished" podID="9cb67476-44eb-41fd-a62d-740ff3df45b8" containerID="406be075d888a7a20bcb33d8dc3ed1bfbc5f7724c7ae2e2c06b36d9ef18e1736" exitCode=0
Sep 16 17:00:38.533992 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.533879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2" event={"ID":"9cb67476-44eb-41fd-a62d-740ff3df45b8","Type":"ContainerDied","Data":"406be075d888a7a20bcb33d8dc3ed1bfbc5f7724c7ae2e2c06b36d9ef18e1736"}
Sep 16 17:00:38.538011 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.537978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:38.553437 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.553395    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"]
Sep 16 17:00:38.553606 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.553455    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6148d487-trffd"]
Sep 16 17:00:38.597053 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.597030    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 17:00:38.620149 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.620118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:38.739473 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.739445    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cb67476-44eb-41fd-a62d-740ff3df45b8-plugins-config-volume\") pod \"9cb67476-44eb-41fd-a62d-740ff3df45b8\" (UID: \"9cb67476-44eb-41fd-a62d-740ff3df45b8\") "
Sep 16 17:00:38.739683 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.739504    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgtbv\" (UniqueName: \"kubernetes.io/projected/9cb67476-44eb-41fd-a62d-740ff3df45b8-kube-api-access-zgtbv\") pod \"9cb67476-44eb-41fd-a62d-740ff3df45b8\" (UID: \"9cb67476-44eb-41fd-a62d-740ff3df45b8\") "
Sep 16 17:00:38.739848 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.739825    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb67476-44eb-41fd-a62d-740ff3df45b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9cb67476-44eb-41fd-a62d-740ff3df45b8" (UID: "9cb67476-44eb-41fd-a62d-740ff3df45b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:00:38.741582 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.741557    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb67476-44eb-41fd-a62d-740ff3df45b8-kube-api-access-zgtbv" (OuterVolumeSpecName: "kube-api-access-zgtbv") pod "9cb67476-44eb-41fd-a62d-740ff3df45b8" (UID: "9cb67476-44eb-41fd-a62d-740ff3df45b8"). InnerVolumeSpecName "kube-api-access-zgtbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:00:38.840866 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.840789    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cb67476-44eb-41fd-a62d-740ff3df45b8-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:00:38.840866 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.840815    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgtbv\" (UniqueName: \"kubernetes.io/projected/9cb67476-44eb-41fd-a62d-740ff3df45b8-kube-api-access-zgtbv\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:00:38.977883 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.977842    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01248bb0-512c-4adc-9497-c6a8b1f43f07" path="/var/lib/kubelet/pods/01248bb0-512c-4adc-9497-c6a8b1f43f07/volumes"
Sep 16 17:00:38.978229 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:38.978209    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e68d17f-a353-4861-8660-702869e5777d" path="/var/lib/kubelet/pods/6e68d17f-a353-4861-8660-702869e5777d/volumes"
Sep 16 17:00:39.003352 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:39.003321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:39.541634 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:39.541606    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"
Sep 16 17:00:39.542036 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:39.541643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2" event={"ID":"9cb67476-44eb-41fd-a62d-740ff3df45b8","Type":"ContainerDied","Data":"797cf5790e9115a90961a5069e4b0df845e97c264a7587a495b2cebdad638983"}
Sep 16 17:00:39.542036 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:39.541702    2577 scope.go:117] "RemoveContainer" containerID="406be075d888a7a20bcb33d8dc3ed1bfbc5f7724c7ae2e2c06b36d9ef18e1736"
Sep 16 17:00:39.562753 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:39.562730    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"]
Sep 16 17:00:39.565265 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:39.565245    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6148d487-848dc4b57-zcwn2"]
Sep 16 17:00:40.977830 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:40.977797    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cb67476-44eb-41fd-a62d-740ff3df45b8" path="/var/lib/kubelet/pods/9cb67476-44eb-41fd-a62d-740ff3df45b8/volumes"
Sep 16 17:00:41.124190 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:41.124153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:41.224756 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:41.224721    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:41.549594 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:41.549543    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:41.698807 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:41.698766    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:42.257900 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:42.257870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:46.784435 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:46.784395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:00:54.386421 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:00:54.386388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:05.901975 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:05.901934    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:15.389342 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:15.389304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:23.498189 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:23.498153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:25.003120 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:25.003029    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:01:25.008467 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:25.006442    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:01:25.181467 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:25.181427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:25.190960 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:25.190933    2577 scope.go:117] "RemoveContainer" containerID="ed3bbac4ab347c7b305eecf1372b72853841bfc870b4237d3dc2c99ee0f0cba0"
Sep 16 17:01:25.243638 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:25.243613    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:29.354140 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:29.354097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:29.454916 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:29.454878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:29.600930 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:29.600899    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:30.800308 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:30.800271    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:30.897881 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:30.897843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:30.998832 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:30.998800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:31.548646 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:31.548599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:31.686022 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:31.685986    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:31.832662 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:31.832566    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:32.987896 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:32.987854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:36.931180 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.931140    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7920-b84f06"]
Sep 16 17:01:36.931668 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.931541    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e68d17f-a353-4861-8660-702869e5777d" containerName="revoke-keys"
Sep 16 17:01:36.931668 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.931553    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e68d17f-a353-4861-8660-702869e5777d" containerName="revoke-keys"
Sep 16 17:01:36.931668 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.931567    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cb67476-44eb-41fd-a62d-740ff3df45b8" containerName="payload-processing"
Sep 16 17:01:36.931668 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.931573    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cb67476-44eb-41fd-a62d-740ff3df45b8" containerName="payload-processing"
Sep 16 17:01:36.931668 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.931652    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e68d17f-a353-4861-8660-702869e5777d" containerName="revoke-keys"
Sep 16 17:01:36.931668 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.931664    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cb67476-44eb-41fd-a62d-740ff3df45b8" containerName="payload-processing"
Sep 16 17:01:36.935882 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.935861    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-b84f06"
Sep 16 17:01:36.938445 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.938420    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rcrsh\""
Sep 16 17:01:36.940237 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.940213    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7920-b84f06"]
Sep 16 17:01:36.979247 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:36.979217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gs86\" (UniqueName: \"kubernetes.io/projected/98aae28e-dd10-47c3-9619-5cd13ab5717f-kube-api-access-2gs86\") pod \"test-curl-7920-b84f06\" (UID: \"98aae28e-dd10-47c3-9619-5cd13ab5717f\") " pod="opendatahub/test-curl-7920-b84f06"
Sep 16 17:01:37.080039 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:37.080003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gs86\" (UniqueName: \"kubernetes.io/projected/98aae28e-dd10-47c3-9619-5cd13ab5717f-kube-api-access-2gs86\") pod \"test-curl-7920-b84f06\" (UID: \"98aae28e-dd10-47c3-9619-5cd13ab5717f\") " pod="opendatahub/test-curl-7920-b84f06"
Sep 16 17:01:37.088071 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:37.088042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gs86\" (UniqueName: \"kubernetes.io/projected/98aae28e-dd10-47c3-9619-5cd13ab5717f-kube-api-access-2gs86\") pod \"test-curl-7920-b84f06\" (UID: \"98aae28e-dd10-47c3-9619-5cd13ab5717f\") " pod="opendatahub/test-curl-7920-b84f06"
Sep 16 17:01:37.246720 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:37.246644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-b84f06"
Sep 16 17:01:37.516475 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:37.516438    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7920-b84f06"]
Sep 16 17:01:37.517951 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:01:37.517918    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98aae28e_dd10_47c3_9619_5cd13ab5717f.slice/crio-29d77adcc48e918789235578503693befd0b1f14be842219452615309282d6f1 WatchSource:0}: Error finding container 29d77adcc48e918789235578503693befd0b1f14be842219452615309282d6f1: Status 404 returned error can't find the container with id 29d77adcc48e918789235578503693befd0b1f14be842219452615309282d6f1
Sep 16 17:01:37.907118 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:37.907081    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7920-b84f06" event={"ID":"98aae28e-dd10-47c3-9619-5cd13ab5717f","Type":"ContainerStarted","Data":"c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4"}
Sep 16 17:01:37.907271 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:37.907129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7920-b84f06" event={"ID":"98aae28e-dd10-47c3-9619-5cd13ab5717f","Type":"ContainerStarted","Data":"29d77adcc48e918789235578503693befd0b1f14be842219452615309282d6f1"}
Sep 16 17:01:37.921174 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:37.921118    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7920-b84f06" podStartSLOduration=1.921103961 podStartE2EDuration="1.921103961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:01:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:01:37.920707045 +0000 UTC m=+1513.417580182" watchObservedRunningTime="2026-09-16 17:01:37.921103961 +0000 UTC m=+1513.417977083"
Sep 16 17:01:38.486866 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:38.486832    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7920-b84f06"]
Sep 16 17:01:38.494511 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:38.494484    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7920-b84f06"]
Sep 16 17:01:38.913323 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:01:38.913295    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7920-b84f06" container="test-curl-7920-b84f06"
Sep 16 17:01:39.148676 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:39.148633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:39.225410 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:39.225320    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:39.778754 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:39.778717    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:39.918819 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:39.918777    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7920-b84f06" podUID="98aae28e-dd10-47c3-9619-5cd13ab5717f" containerName="test-curl-7920-b84f06" containerID="cri-o://c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4" gracePeriod=2
Sep 16 17:01:42.099009 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.098985    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-b84f06"
Sep 16 17:01:42.126309 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.126283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gs86\" (UniqueName: \"kubernetes.io/projected/98aae28e-dd10-47c3-9619-5cd13ab5717f-kube-api-access-2gs86\") pod \"98aae28e-dd10-47c3-9619-5cd13ab5717f\" (UID: \"98aae28e-dd10-47c3-9619-5cd13ab5717f\") "
Sep 16 17:01:42.128214 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.128192    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98aae28e-dd10-47c3-9619-5cd13ab5717f-kube-api-access-2gs86" (OuterVolumeSpecName: "kube-api-access-2gs86") pod "98aae28e-dd10-47c3-9619-5cd13ab5717f" (UID: "98aae28e-dd10-47c3-9619-5cd13ab5717f"). InnerVolumeSpecName "kube-api-access-2gs86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:01:42.227668 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.227591    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gs86\" (UniqueName: \"kubernetes.io/projected/98aae28e-dd10-47c3-9619-5cd13ab5717f-kube-api-access-2gs86\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:01:42.381109 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.381076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:42.937628 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.937596    2577 generic.go:356] "Generic (PLEG): container finished" podID="98aae28e-dd10-47c3-9619-5cd13ab5717f" containerID="c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4" exitCode=137
Sep 16 17:01:42.937818 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.937648    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-b84f06"
Sep 16 17:01:42.937818 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.937693    2577 scope.go:117] "RemoveContainer" containerID="c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4"
Sep 16 17:01:42.948056 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.948041    2577 scope.go:117] "RemoveContainer" containerID="c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4"
Sep 16 17:01:42.948384 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:01:42.948316    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4\": container with ID starting with c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4 not found: ID does not exist" containerID="c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4"
Sep 16 17:01:42.948456 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.948396    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4"} err="failed to get container status \"c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4\": rpc error: code = NotFound desc = could not find container \"c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4\": container with ID starting with c619bdd419c59d44e7345517de615b98ebc1bcad83c5e232def47d273b1f0cf4 not found: ID does not exist"
Sep 16 17:01:42.948968 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:01:42.948945    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7920-b84f06\" is forbidden: User \"system:node:ip-10-0-139-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-51.ec2.internal' and this object" podUID="98aae28e-dd10-47c3-9619-5cd13ab5717f" pod="opendatahub/test-curl-7920-b84f06"
Sep 16 17:01:42.977285 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:42.977262    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98aae28e-dd10-47c3-9619-5cd13ab5717f" path="/var/lib/kubelet/pods/98aae28e-dd10-47c3-9619-5cd13ab5717f/volumes"
Sep 16 17:01:45.771880 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:45.771843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:45.876430 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:45.876389    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:45.971850 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:45.971813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:55.516220 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:55.516128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:56.437205 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:56.437166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:01:56.638675 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:01:56.638643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:00.694322 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:00.694270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:17.539312 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:17.539277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:20.682638 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:20.682600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:23.224211 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:23.224176    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:23.326357 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:23.326325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:23.555827 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:23.555735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:23.854588 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:23.854479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:24.547285 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:24.547251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:25.219505 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:25.219473    2577 scope.go:117] "RemoveContainer" containerID="1bc4d12558810489630e21fb2de4a02e29e49a15cc9fcb3776aba743c9808ea3"
Sep 16 17:02:25.228966 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:25.228947    2577 scope.go:117] "RemoveContainer" containerID="7e062ad61f3e49fc752f3906722143a4b819755520f2f29f8749d8d988e0a46f"
Sep 16 17:02:27.402266 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:27.402232    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:27.578140 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:27.578098    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:27.675058 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:27.674970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:28.405845 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:28.405802    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:29.434789 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:29.434753    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:30.383244 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:30.383211    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:31.282589 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:31.282524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:31.380424 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:31.380393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:31.642693 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:31.642654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:31.924212 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:31.924122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:32.025782 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.025741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:32.035827 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.035801    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"]
Sep 16 17:02:32.036250 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.036236    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98aae28e-dd10-47c3-9619-5cd13ab5717f" containerName="test-curl-7920-b84f06"
Sep 16 17:02:32.036294 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.036253    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="98aae28e-dd10-47c3-9619-5cd13ab5717f" containerName="test-curl-7920-b84f06"
Sep 16 17:02:32.036334 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.036323    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="98aae28e-dd10-47c3-9619-5cd13ab5717f" containerName="test-curl-7920-b84f06"
Sep 16 17:02:32.039643 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.039628    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:32.042356 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.042335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-9359a3-dockercfg-znfnt\""
Sep 16 17:02:32.042486 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.042378    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-9359a3\""
Sep 16 17:02:32.050163 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.050141    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"]
Sep 16 17:02:32.061379 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.061355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c98f0c5-7aee-4ed2-989a-93547b78b03f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p\" (UID: \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:32.061505 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.061418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6rhw\" (UniqueName: \"kubernetes.io/projected/9c98f0c5-7aee-4ed2-989a-93547b78b03f-kube-api-access-s6rhw\") pod \"payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p\" (UID: \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:32.066243 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.066222    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"]
Sep 16 17:02:32.070034 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.070019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:32.082042 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.082022    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"]
Sep 16 17:02:32.162786 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.162753    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039b2915-0615-4c35-b18c-ca9ec6a7b854-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv\" (UID: \"039b2915-0615-4c35-b18c-ca9ec6a7b854\") " pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:32.162956 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.162861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c98f0c5-7aee-4ed2-989a-93547b78b03f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p\" (UID: \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:32.162956 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.162916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6rhw\" (UniqueName: \"kubernetes.io/projected/9c98f0c5-7aee-4ed2-989a-93547b78b03f-kube-api-access-s6rhw\") pod \"payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p\" (UID: \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:32.162956 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.162944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfvdv\" (UniqueName: \"kubernetes.io/projected/039b2915-0615-4c35-b18c-ca9ec6a7b854-kube-api-access-jfvdv\") pod \"payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv\" (UID: \"039b2915-0615-4c35-b18c-ca9ec6a7b854\") " pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:32.163659 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.163635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c98f0c5-7aee-4ed2-989a-93547b78b03f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p\" (UID: \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:32.171391 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.171364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6rhw\" (UniqueName: \"kubernetes.io/projected/9c98f0c5-7aee-4ed2-989a-93547b78b03f-kube-api-access-s6rhw\") pod \"payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p\" (UID: \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:32.263742 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.263634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039b2915-0615-4c35-b18c-ca9ec6a7b854-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv\" (UID: \"039b2915-0615-4c35-b18c-ca9ec6a7b854\") " pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:32.263742 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.263728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfvdv\" (UniqueName: \"kubernetes.io/projected/039b2915-0615-4c35-b18c-ca9ec6a7b854-kube-api-access-jfvdv\") pod \"payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv\" (UID: \"039b2915-0615-4c35-b18c-ca9ec6a7b854\") " pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:32.264466 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.264437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039b2915-0615-4c35-b18c-ca9ec6a7b854-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv\" (UID: \"039b2915-0615-4c35-b18c-ca9ec6a7b854\") " pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:32.271651 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.271627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfvdv\" (UniqueName: \"kubernetes.io/projected/039b2915-0615-4c35-b18c-ca9ec6a7b854-kube-api-access-jfvdv\") pod \"payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv\" (UID: \"039b2915-0615-4c35-b18c-ca9ec6a7b854\") " pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:32.350495 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.350463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:32.380709 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.380684    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:32.627612 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.627582    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"]
Sep 16 17:02:32.628368 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:02:32.628311    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c98f0c5_7aee_4ed2_989a_93547b78b03f.slice/crio-1b88a7a89ee6a57048971f278f002a71505f226ee5b82a20e52f102067414af5 WatchSource:0}: Error finding container 1b88a7a89ee6a57048971f278f002a71505f226ee5b82a20e52f102067414af5: Status 404 returned error can't find the container with id 1b88a7a89ee6a57048971f278f002a71505f226ee5b82a20e52f102067414af5
Sep 16 17:02:32.671104 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:02:32.671080    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039b2915_0615_4c35_b18c_ca9ec6a7b854.slice/crio-2c3acfb64b0722089f74918d748b6c67452222636a71aff71cb7dc5bcde843d6 WatchSource:0}: Error finding container 2c3acfb64b0722089f74918d748b6c67452222636a71aff71cb7dc5bcde843d6: Status 404 returned error can't find the container with id 2c3acfb64b0722089f74918d748b6c67452222636a71aff71cb7dc5bcde843d6
Sep 16 17:02:32.671240 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:32.671217    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"]
Sep 16 17:02:33.245997 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:33.245956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv" event={"ID":"039b2915-0615-4c35-b18c-ca9ec6a7b854","Type":"ContainerStarted","Data":"8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127"}
Sep 16 17:02:33.246179 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:33.246017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv" event={"ID":"039b2915-0615-4c35-b18c-ca9ec6a7b854","Type":"ContainerStarted","Data":"2c3acfb64b0722089f74918d748b6c67452222636a71aff71cb7dc5bcde843d6"}
Sep 16 17:02:33.246179 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:33.246049    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:33.247844 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:33.247822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p" event={"ID":"9c98f0c5-7aee-4ed2-989a-93547b78b03f","Type":"ContainerStarted","Data":"41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89"}
Sep 16 17:02:33.247964 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:33.247872    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:33.247964 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:33.247890    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p" event={"ID":"9c98f0c5-7aee-4ed2-989a-93547b78b03f","Type":"ContainerStarted","Data":"1b88a7a89ee6a57048971f278f002a71505f226ee5b82a20e52f102067414af5"}
Sep 16 17:02:33.261607 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:33.261562    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv" podStartSLOduration=1.261544552 podStartE2EDuration="1.261544552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:02:33.260577842 +0000 UTC m=+1568.757450966" watchObservedRunningTime="2026-09-16 17:02:33.261544552 +0000 UTC m=+1568.758417676"
Sep 16 17:02:33.279070 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:33.279033    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p" podStartSLOduration=1.279020516 podStartE2EDuration="1.279020516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:02:33.276175811 +0000 UTC m=+1568.773048934" watchObservedRunningTime="2026-09-16 17:02:33.279020516 +0000 UTC m=+1568.775893640"
Sep 16 17:02:44.255970 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:44.255934    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:02:44.256353 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:44.256020    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:02:55.141781 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:55.141739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:56.071697 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:56.071660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:57.140742 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:57.140702    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:57.608041 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:57.607947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:58.080209 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:58.080175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:58.239280 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:58.239239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:58.466019 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:58.465985    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:58.705005 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:58.704968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:02:59.743681 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:02:59.743635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:00.952797 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:00.952760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:02.012673 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:02.012633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:02.236244 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:02.236212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:03.842429 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:03.842392    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:05.351775 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:05.351730    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:06.641006 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:06.640972    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:07.745395 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:07.745358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:08.618014 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:08.617974    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:09.737731 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:09.737693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:09.838958 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:09.838919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:10.317982 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.317942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:10.414446 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.414412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:10.485815 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.485781    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"]
Sep 16 17:03:10.490420 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.490400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:10.494173 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.494147    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-4acea1\""
Sep 16 17:03:10.494288 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.494191    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-4acea1-dockercfg-9xp42\""
Sep 16 17:03:10.503542 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.503505    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"]
Sep 16 17:03:10.533266 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.533242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:10.545427 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.545404    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"]
Sep 16 17:03:10.548769 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.548753    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:10.575705 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.575656    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"]
Sep 16 17:03:10.621105 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.621080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd7a9301-cd66-413a-abf1-f3c52019a058-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4\" (UID: \"fd7a9301-cd66-413a-abf1-f3c52019a058\") " pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:10.621199 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.621175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w4hw\" (UniqueName: \"kubernetes.io/projected/fd7a9301-cd66-413a-abf1-f3c52019a058-kube-api-access-6w4hw\") pod \"payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4\" (UID: \"fd7a9301-cd66-413a-abf1-f3c52019a058\") " pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:10.721799 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.721771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd7a9301-cd66-413a-abf1-f3c52019a058-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4\" (UID: \"fd7a9301-cd66-413a-abf1-f3c52019a058\") " pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:10.721953 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.721844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47fdt\" (UniqueName: \"kubernetes.io/projected/35f0869e-367c-4c6c-a185-f9baaf3748e0-kube-api-access-47fdt\") pod \"payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5\" (UID: \"35f0869e-367c-4c6c-a185-f9baaf3748e0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:10.721953 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.721867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w4hw\" (UniqueName: \"kubernetes.io/projected/fd7a9301-cd66-413a-abf1-f3c52019a058-kube-api-access-6w4hw\") pod \"payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4\" (UID: \"fd7a9301-cd66-413a-abf1-f3c52019a058\") " pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:10.721953 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.721898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0869e-367c-4c6c-a185-f9baaf3748e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5\" (UID: \"35f0869e-367c-4c6c-a185-f9baaf3748e0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:10.722469 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.722449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd7a9301-cd66-413a-abf1-f3c52019a058-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4\" (UID: \"fd7a9301-cd66-413a-abf1-f3c52019a058\") " pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:10.736421 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.736390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w4hw\" (UniqueName: \"kubernetes.io/projected/fd7a9301-cd66-413a-abf1-f3c52019a058-kube-api-access-6w4hw\") pod \"payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4\" (UID: \"fd7a9301-cd66-413a-abf1-f3c52019a058\") " pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:10.802071 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.802035    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:10.822626 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.822599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47fdt\" (UniqueName: \"kubernetes.io/projected/35f0869e-367c-4c6c-a185-f9baaf3748e0-kube-api-access-47fdt\") pod \"payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5\" (UID: \"35f0869e-367c-4c6c-a185-f9baaf3748e0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:10.822756 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.822652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0869e-367c-4c6c-a185-f9baaf3748e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5\" (UID: \"35f0869e-367c-4c6c-a185-f9baaf3748e0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:10.823269 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.823244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0869e-367c-4c6c-a185-f9baaf3748e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5\" (UID: \"35f0869e-367c-4c6c-a185-f9baaf3748e0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:10.838937 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.838875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47fdt\" (UniqueName: \"kubernetes.io/projected/35f0869e-367c-4c6c-a185-f9baaf3748e0-kube-api-access-47fdt\") pod \"payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5\" (UID: \"35f0869e-367c-4c6c-a185-f9baaf3748e0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:10.858910 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:10.858882    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:11.081967 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.081935    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"]
Sep 16 17:03:11.083000 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:03:11.082974    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7a9301_cd66_413a_abf1_f3c52019a058.slice/crio-ca300eba5f3b400e77940d6564de6520a60059f06dc361da179bfa7fe39457e3 WatchSource:0}: Error finding container ca300eba5f3b400e77940d6564de6520a60059f06dc361da179bfa7fe39457e3: Status 404 returned error can't find the container with id ca300eba5f3b400e77940d6564de6520a60059f06dc361da179bfa7fe39457e3
Sep 16 17:03:11.147343 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.147320    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"]
Sep 16 17:03:11.147660 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:03:11.147637    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35f0869e_367c_4c6c_a185_f9baaf3748e0.slice/crio-0d4f8fe8c9e406c20e104292c8cc6a20407b34c9a8423b3a5e3f66ff7eaec6d1 WatchSource:0}: Error finding container 0d4f8fe8c9e406c20e104292c8cc6a20407b34c9a8423b3a5e3f66ff7eaec6d1: Status 404 returned error can't find the container with id 0d4f8fe8c9e406c20e104292c8cc6a20407b34c9a8423b3a5e3f66ff7eaec6d1
Sep 16 17:03:11.483379 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.483339    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5" event={"ID":"35f0869e-367c-4c6c-a185-f9baaf3748e0","Type":"ContainerStarted","Data":"e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7"}
Sep 16 17:03:11.483582 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.483406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5" event={"ID":"35f0869e-367c-4c6c-a185-f9baaf3748e0","Type":"ContainerStarted","Data":"0d4f8fe8c9e406c20e104292c8cc6a20407b34c9a8423b3a5e3f66ff7eaec6d1"}
Sep 16 17:03:11.483582 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.483518    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:11.484999 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.484979    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4" event={"ID":"fd7a9301-cd66-413a-abf1-f3c52019a058","Type":"ContainerStarted","Data":"8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75"}
Sep 16 17:03:11.485048 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.485019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4" event={"ID":"fd7a9301-cd66-413a-abf1-f3c52019a058","Type":"ContainerStarted","Data":"ca300eba5f3b400e77940d6564de6520a60059f06dc361da179bfa7fe39457e3"}
Sep 16 17:03:11.485048 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.485039    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:11.503678 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.503632    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5" podStartSLOduration=1.5036189709999999 podStartE2EDuration="1.503618971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:03:11.500802338 +0000 UTC m=+1606.997675463" watchObservedRunningTime="2026-09-16 17:03:11.503618971 +0000 UTC m=+1607.000492132"
Sep 16 17:03:11.519785 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:11.519736    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4" podStartSLOduration=1.5197203190000002 podStartE2EDuration="1.519720319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:03:11.518224715 +0000 UTC m=+1607.015097839" watchObservedRunningTime="2026-09-16 17:03:11.519720319 +0000 UTC m=+1607.016593444"
Sep 16 17:03:14.917486 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:14.917448    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7920-dc181b"]
Sep 16 17:03:14.921749 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:14.921731    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-dc181b"
Sep 16 17:03:14.925169 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:14.925146    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rcrsh\""
Sep 16 17:03:14.926887 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:14.926861    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7920-dc181b"]
Sep 16 17:03:15.061407 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:15.061358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hmql\" (UniqueName: \"kubernetes.io/projected/20606a78-a256-48b8-ae03-99dd42d0d4f8-kube-api-access-2hmql\") pod \"test-curl-7920-dc181b\" (UID: \"20606a78-a256-48b8-ae03-99dd42d0d4f8\") " pod="opendatahub/test-curl-7920-dc181b"
Sep 16 17:03:15.162722 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:15.162672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hmql\" (UniqueName: \"kubernetes.io/projected/20606a78-a256-48b8-ae03-99dd42d0d4f8-kube-api-access-2hmql\") pod \"test-curl-7920-dc181b\" (UID: \"20606a78-a256-48b8-ae03-99dd42d0d4f8\") " pod="opendatahub/test-curl-7920-dc181b"
Sep 16 17:03:15.172064 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:15.172001    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hmql\" (UniqueName: \"kubernetes.io/projected/20606a78-a256-48b8-ae03-99dd42d0d4f8-kube-api-access-2hmql\") pod \"test-curl-7920-dc181b\" (UID: \"20606a78-a256-48b8-ae03-99dd42d0d4f8\") " pod="opendatahub/test-curl-7920-dc181b"
Sep 16 17:03:15.233547 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:15.233506    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-dc181b"
Sep 16 17:03:15.507174 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:15.507143    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7920-dc181b"]
Sep 16 17:03:15.509847 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:03:15.509789    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20606a78_a256_48b8_ae03_99dd42d0d4f8.slice/crio-952c9b74ec56054341068cebe9b22313d98a08ec3ae96045f59b3d8752eb497f WatchSource:0}: Error finding container 952c9b74ec56054341068cebe9b22313d98a08ec3ae96045f59b3d8752eb497f: Status 404 returned error can't find the container with id 952c9b74ec56054341068cebe9b22313d98a08ec3ae96045f59b3d8752eb497f
Sep 16 17:03:16.521288 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:16.521252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7920-dc181b" event={"ID":"20606a78-a256-48b8-ae03-99dd42d0d4f8","Type":"ContainerStarted","Data":"c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b"}
Sep 16 17:03:16.521813 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:16.521314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7920-dc181b" event={"ID":"20606a78-a256-48b8-ae03-99dd42d0d4f8","Type":"ContainerStarted","Data":"952c9b74ec56054341068cebe9b22313d98a08ec3ae96045f59b3d8752eb497f"}
Sep 16 17:03:16.541329 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:16.541278    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7920-dc181b" podStartSLOduration=2.54126573 podStartE2EDuration="2.54126573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:03:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:03:16.538509534 +0000 UTC m=+1612.035382659" watchObservedRunningTime="2026-09-16 17:03:16.54126573 +0000 UTC m=+1612.038138853"
Sep 16 17:03:17.194935 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.194852    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7920-dc181b"]
Sep 16 17:03:17.202478 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.202445    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7920-dc181b"]
Sep 16 17:03:17.315648 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.315552    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7920-99b4a8"]
Sep 16 17:03:17.316120 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.316100    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20606a78-a256-48b8-ae03-99dd42d0d4f8" containerName="test-curl-7920-dc181b"
Sep 16 17:03:17.316226 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.316122    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="20606a78-a256-48b8-ae03-99dd42d0d4f8" containerName="test-curl-7920-dc181b"
Sep 16 17:03:17.316278 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.316240    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="20606a78-a256-48b8-ae03-99dd42d0d4f8" containerName="test-curl-7920-dc181b"
Sep 16 17:03:17.319613 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.319593    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:17.326924 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.326895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7920-99b4a8"]
Sep 16 17:03:17.379905 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.379871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdqr4\" (UniqueName: \"kubernetes.io/projected/de642d5e-305c-4cc9-8f15-94263be0fac0-kube-api-access-mdqr4\") pod \"test-curl-7920-99b4a8\" (UID: \"de642d5e-305c-4cc9-8f15-94263be0fac0\") " pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:17.481180 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.481147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdqr4\" (UniqueName: \"kubernetes.io/projected/de642d5e-305c-4cc9-8f15-94263be0fac0-kube-api-access-mdqr4\") pod \"test-curl-7920-99b4a8\" (UID: \"de642d5e-305c-4cc9-8f15-94263be0fac0\") " pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:17.489661 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.489628    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdqr4\" (UniqueName: \"kubernetes.io/projected/de642d5e-305c-4cc9-8f15-94263be0fac0-kube-api-access-mdqr4\") pod \"test-curl-7920-99b4a8\" (UID: \"de642d5e-305c-4cc9-8f15-94263be0fac0\") " pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:17.527872 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:17.527845    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7920-dc181b" container="test-curl-7920-dc181b"
Sep 16 17:03:17.631173 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.631093    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:17.908417 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:17.908389    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7920-99b4a8"]
Sep 16 17:03:17.910111 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:03:17.910085    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde642d5e_305c_4cc9_8f15_94263be0fac0.slice/crio-068875cbe5830d06b4a9f0c120e7421d55e8f411ab93a3e553098c558470f3ea WatchSource:0}: Error finding container 068875cbe5830d06b4a9f0c120e7421d55e8f411ab93a3e553098c558470f3ea: Status 404 returned error can't find the container with id 068875cbe5830d06b4a9f0c120e7421d55e8f411ab93a3e553098c558470f3ea
Sep 16 17:03:18.536196 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:18.536152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7920-99b4a8" event={"ID":"de642d5e-305c-4cc9-8f15-94263be0fac0","Type":"ContainerStarted","Data":"09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb"}
Sep 16 17:03:18.536196 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:18.536203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7920-99b4a8" event={"ID":"de642d5e-305c-4cc9-8f15-94263be0fac0","Type":"ContainerStarted","Data":"068875cbe5830d06b4a9f0c120e7421d55e8f411ab93a3e553098c558470f3ea"}
Sep 16 17:03:18.536670 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:18.536321    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7920-dc181b" podUID="20606a78-a256-48b8-ae03-99dd42d0d4f8" containerName="test-curl-7920-dc181b" containerID="cri-o://c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b" gracePeriod=2
Sep 16 17:03:18.561806 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:18.561764    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7920-99b4a8" podStartSLOduration=1.561748808 podStartE2EDuration="1.561748808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:03:18.560647081 +0000 UTC m=+1614.057520206" watchObservedRunningTime="2026-09-16 17:03:18.561748808 +0000 UTC m=+1614.058621931"
Sep 16 17:03:18.564656 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:18.564630    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7920-dc181b\" is forbidden: User \"system:node:ip-10-0-139-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-51.ec2.internal' and this object" podUID="20606a78-a256-48b8-ae03-99dd42d0d4f8" pod="opendatahub/test-curl-7920-dc181b"
Sep 16 17:03:19.096226 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:19.096177    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:19.221775 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:19.221740    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7920-99b4a8"]
Sep 16 17:03:19.226265 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:19.226227    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7920-99b4a8"]
Sep 16 17:03:19.543172 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:19.543139    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7920-99b4a8" container="test-curl-7920-99b4a8"
Sep 16 17:03:20.553904 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:20.553864    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7920-99b4a8" podUID="de642d5e-305c-4cc9-8f15-94263be0fac0" containerName="test-curl-7920-99b4a8" containerID="cri-o://09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb" gracePeriod=2
Sep 16 17:03:20.556311 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:20.556282    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7920-99b4a8\" is forbidden: User \"system:node:ip-10-0-139-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-51.ec2.internal' and this object" podUID="de642d5e-305c-4cc9-8f15-94263be0fac0" pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:20.722262 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:20.722236    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-dc181b"
Sep 16 17:03:20.724677 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:20.724649    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7920-99b4a8\" is forbidden: User \"system:node:ip-10-0-139-51.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-51.ec2.internal' and this object" podUID="de642d5e-305c-4cc9-8f15-94263be0fac0" pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:20.812518 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:20.812441    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hmql\" (UniqueName: \"kubernetes.io/projected/20606a78-a256-48b8-ae03-99dd42d0d4f8-kube-api-access-2hmql\") pod \"20606a78-a256-48b8-ae03-99dd42d0d4f8\" (UID: \"20606a78-a256-48b8-ae03-99dd42d0d4f8\") "
Sep 16 17:03:20.814503 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:20.814472    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20606a78-a256-48b8-ae03-99dd42d0d4f8-kube-api-access-2hmql" (OuterVolumeSpecName: "kube-api-access-2hmql") pod "20606a78-a256-48b8-ae03-99dd42d0d4f8" (UID: "20606a78-a256-48b8-ae03-99dd42d0d4f8"). InnerVolumeSpecName "kube-api-access-2hmql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:03:20.913437 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:20.913409    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hmql\" (UniqueName: \"kubernetes.io/projected/20606a78-a256-48b8-ae03-99dd42d0d4f8-kube-api-access-2hmql\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:03:20.977747 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:20.977716    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20606a78-a256-48b8-ae03-99dd42d0d4f8" path="/var/lib/kubelet/pods/20606a78-a256-48b8-ae03-99dd42d0d4f8/volumes"
Sep 16 17:03:21.563611 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:21.563577    2577 generic.go:356] "Generic (PLEG): container finished" podID="20606a78-a256-48b8-ae03-99dd42d0d4f8" containerID="c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b" exitCode=137
Sep 16 17:03:21.564093 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:21.563733    2577 scope.go:117] "RemoveContainer" containerID="c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b"
Sep 16 17:03:21.564093 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:21.563851    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-dc181b"
Sep 16 17:03:21.574221 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:21.574195    2577 scope.go:117] "RemoveContainer" containerID="c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b"
Sep 16 17:03:21.574549 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:21.574515    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b\": container with ID starting with c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b not found: ID does not exist" containerID="c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b"
Sep 16 17:03:21.574647 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:21.574560    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b"} err="failed to get container status \"c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b\": rpc error: code = NotFound desc = could not find container \"c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b\": container with ID starting with c4e86fb14c5a3397d9427efe01cf9ae863196539b383ed57d8c10a88e1628c9b not found: ID does not exist"
Sep 16 17:03:21.619290 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:21.619265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:22.493390 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.493361    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:03:22.493687 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.493664    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:03:22.726295 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.726251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:22.747937 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.747884    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:22.829866 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.829835    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdqr4\" (UniqueName: \"kubernetes.io/projected/de642d5e-305c-4cc9-8f15-94263be0fac0-kube-api-access-mdqr4\") pod \"de642d5e-305c-4cc9-8f15-94263be0fac0\" (UID: \"de642d5e-305c-4cc9-8f15-94263be0fac0\") "
Sep 16 17:03:22.831843 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.831821    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de642d5e-305c-4cc9-8f15-94263be0fac0-kube-api-access-mdqr4" (OuterVolumeSpecName: "kube-api-access-mdqr4") pod "de642d5e-305c-4cc9-8f15-94263be0fac0" (UID: "de642d5e-305c-4cc9-8f15-94263be0fac0"). InnerVolumeSpecName "kube-api-access-mdqr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:03:22.839917 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.839892    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:22.930519 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.930494    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdqr4\" (UniqueName: \"kubernetes.io/projected/de642d5e-305c-4cc9-8f15-94263be0fac0-kube-api-access-mdqr4\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:03:22.978116 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:22.978083    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de642d5e-305c-4cc9-8f15-94263be0fac0" path="/var/lib/kubelet/pods/de642d5e-305c-4cc9-8f15-94263be0fac0/volumes"
Sep 16 17:03:23.578854 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:23.578817    2577 generic.go:356] "Generic (PLEG): container finished" podID="de642d5e-305c-4cc9-8f15-94263be0fac0" containerID="09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb" exitCode=137
Sep 16 17:03:23.579062 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:23.578861    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7920-99b4a8"
Sep 16 17:03:23.579062 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:23.578893    2577 scope.go:117] "RemoveContainer" containerID="09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb"
Sep 16 17:03:23.587915 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:23.587893    2577 scope.go:117] "RemoveContainer" containerID="09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb"
Sep 16 17:03:23.588183 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:23.588165    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb\": container with ID starting with 09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb not found: ID does not exist" containerID="09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb"
Sep 16 17:03:23.588245 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:23.588189    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb"} err="failed to get container status \"09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb\": rpc error: code = NotFound desc = could not find container \"09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb\": container with ID starting with 09769c84d6912e17f39c9d5c0fe69bd1caf7ddaaf6192e4d357273d9bb213ddb not found: ID does not exist"
Sep 16 17:03:24.326894 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:24.326859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:25.257830 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:25.257801    2577 scope.go:117] "RemoveContainer" containerID="37514886315e7c042b4367606a32ad03f10fbf8c7b82e6b935e4528196efab03"
Sep 16 17:03:25.266669 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:25.266651    2577 scope.go:117] "RemoveContainer" containerID="8606c6a214311a7c22077f43666858776d2dde80d4aa74f4a87d924bb10bb12a"
Sep 16 17:03:25.274893 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:25.274870    2577 scope.go:117] "RemoveContainer" containerID="57efb907b4dcc413bdf22a648fceb10c4c5fe64f4158d80cf6b84e3ab517fa96"
Sep 16 17:03:30.453835 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:30.453799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:30.916140 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:30.916095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:32.640436 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:32.640402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:33.631463 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:33.631431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:35.022742 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:35.022686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:35.416145 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:35.416108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:36.409467 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:36.409435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:36.953761 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:36.953728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:37.207927 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:37.207843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:38.180630 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:38.180584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:39.014615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.014581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:39.834639 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.834596    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"]
Sep 16 17:03:39.835063 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.835049    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de642d5e-305c-4cc9-8f15-94263be0fac0" containerName="test-curl-7920-99b4a8"
Sep 16 17:03:39.835063 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.835064    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="de642d5e-305c-4cc9-8f15-94263be0fac0" containerName="test-curl-7920-99b4a8"
Sep 16 17:03:39.835168 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.835159    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="de642d5e-305c-4cc9-8f15-94263be0fac0" containerName="test-curl-7920-99b4a8"
Sep 16 17:03:39.838850 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.838829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"
Sep 16 17:03:39.843491 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.843469    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:03:39.843621 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.843467    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:03:39.851342 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.851314    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"]
Sep 16 17:03:39.881227 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.881200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb8d763-ebbc-44c9-9422-2fc89385a42b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd\" (UID: \"afb8d763-ebbc-44c9-9422-2fc89385a42b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"
Sep 16 17:03:39.981947 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.981904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb8d763-ebbc-44c9-9422-2fc89385a42b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd\" (UID: \"afb8d763-ebbc-44c9-9422-2fc89385a42b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"
Sep 16 17:03:39.982552 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:39.982510    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb8d763-ebbc-44c9-9422-2fc89385a42b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd\" (UID: \"afb8d763-ebbc-44c9-9422-2fc89385a42b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"
Sep 16 17:03:40.150846 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:40.150814    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"
Sep 16 17:03:40.325503 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:40.325470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:40.430093 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:40.430067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"]
Sep 16 17:03:40.430939 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:03:40.430909    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb8d763_ebbc_44c9_9422_2fc89385a42b.slice/crio-81933f8cd73828cd0b1b510d132cd575bc72f25c035bf938ee616704fa0ed9d4 WatchSource:0}: Error finding container 81933f8cd73828cd0b1b510d132cd575bc72f25c035bf938ee616704fa0ed9d4: Status 404 returned error can't find the container with id 81933f8cd73828cd0b1b510d132cd575bc72f25c035bf938ee616704fa0ed9d4
Sep 16 17:03:40.683707 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:40.683621    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd" event={"ID":"afb8d763-ebbc-44c9-9422-2fc89385a42b","Type":"ContainerStarted","Data":"036aa29503d314e0a07e645fb003be1ec6d6527d56d07b1630f88f78a9034798"}
Sep 16 17:03:40.683707 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:40.683679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd" event={"ID":"afb8d763-ebbc-44c9-9422-2fc89385a42b","Type":"ContainerStarted","Data":"81933f8cd73828cd0b1b510d132cd575bc72f25c035bf938ee616704fa0ed9d4"}
Sep 16 17:03:40.721218 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:40.721172    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd" podStartSLOduration=1.721156334 podStartE2EDuration="1.721156334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:03:40.719598197 +0000 UTC m=+1636.216471322" watchObservedRunningTime="2026-09-16 17:03:40.721156334 +0000 UTC m=+1636.218029457"
Sep 16 17:03:41.342905 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:41.342828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:41.690988 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:41.690941    2577 generic.go:356] "Generic (PLEG): container finished" podID="afb8d763-ebbc-44c9-9422-2fc89385a42b" containerID="036aa29503d314e0a07e645fb003be1ec6d6527d56d07b1630f88f78a9034798" exitCode=0
Sep 16 17:03:41.691178 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:41.691013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd" event={"ID":"afb8d763-ebbc-44c9-9422-2fc89385a42b","Type":"ContainerDied","Data":"036aa29503d314e0a07e645fb003be1ec6d6527d56d07b1630f88f78a9034798"}
Sep 16 17:03:42.198183 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:42.198145    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:42.831180 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:42.831158    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"
Sep 16 17:03:42.907899 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:42.907868    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb8d763-ebbc-44c9-9422-2fc89385a42b-maas-api-service-ca\") pod \"afb8d763-ebbc-44c9-9422-2fc89385a42b\" (UID: \"afb8d763-ebbc-44c9-9422-2fc89385a42b\") "
Sep 16 17:03:42.908241 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:42.908219    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afb8d763-ebbc-44c9-9422-2fc89385a42b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "afb8d763-ebbc-44c9-9422-2fc89385a42b" (UID: "afb8d763-ebbc-44c9-9422-2fc89385a42b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:03:43.009295 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:43.009268    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/afb8d763-ebbc-44c9-9422-2fc89385a42b-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:03:43.077007 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:43.076968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:43.705167 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:43.705137    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"
Sep 16 17:03:43.705167 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:43.705171    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd" event={"ID":"afb8d763-ebbc-44c9-9422-2fc89385a42b","Type":"ContainerDied","Data":"81933f8cd73828cd0b1b510d132cd575bc72f25c035bf938ee616704fa0ed9d4"}
Sep 16 17:03:43.705444 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:43.705218    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81933f8cd73828cd0b1b510d132cd575bc72f25c035bf938ee616704fa0ed9d4"
Sep 16 17:03:43.854643 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:43.854606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:44.730839 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:44.730797    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:45.808782 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:45.808733    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:50.087545 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:50.087487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:55.735662 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.735630    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:55.758094 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.758061    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"]
Sep 16 17:03:55.758364 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.758317    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv" podUID="039b2915-0615-4c35-b18c-ca9ec6a7b854" containerName="payload-processing" containerID="cri-o://8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127" gracePeriod=30
Sep 16 17:03:55.768721 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.768485    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"]
Sep 16 17:03:55.769104 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.769074    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p" podUID="9c98f0c5-7aee-4ed2-989a-93547b78b03f" containerName="payload-pre-processing" containerID="cri-o://41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89" gracePeriod=30
Sep 16 17:03:55.806116 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.806075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:55.907636 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.907597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:55.943827 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.943794    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"]
Sep 16 17:03:55.948905 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.948880    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-9359a3-vcvjd"]
Sep 16 17:03:55.987036 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:55.986960    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:56.038261 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.038239    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:03:56.041178 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.041158    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:03:56.133002 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.132960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c98f0c5-7aee-4ed2-989a-93547b78b03f-plugins-config-volume\") pod \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\" (UID: \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\") "
Sep 16 17:03:56.133196 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.132996    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039b2915-0615-4c35-b18c-ca9ec6a7b854-plugins-config-volume\") pod \"039b2915-0615-4c35-b18c-ca9ec6a7b854\" (UID: \"039b2915-0615-4c35-b18c-ca9ec6a7b854\") "
Sep 16 17:03:56.133196 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.133135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfvdv\" (UniqueName: \"kubernetes.io/projected/039b2915-0615-4c35-b18c-ca9ec6a7b854-kube-api-access-jfvdv\") pod \"039b2915-0615-4c35-b18c-ca9ec6a7b854\" (UID: \"039b2915-0615-4c35-b18c-ca9ec6a7b854\") "
Sep 16 17:03:56.133316 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.133220    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6rhw\" (UniqueName: \"kubernetes.io/projected/9c98f0c5-7aee-4ed2-989a-93547b78b03f-kube-api-access-s6rhw\") pod \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\" (UID: \"9c98f0c5-7aee-4ed2-989a-93547b78b03f\") "
Sep 16 17:03:56.133522 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.133428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c98f0c5-7aee-4ed2-989a-93547b78b03f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c98f0c5-7aee-4ed2-989a-93547b78b03f" (UID: "9c98f0c5-7aee-4ed2-989a-93547b78b03f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:03:56.133522 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.133480    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/039b2915-0615-4c35-b18c-ca9ec6a7b854-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "039b2915-0615-4c35-b18c-ca9ec6a7b854" (UID: "039b2915-0615-4c35-b18c-ca9ec6a7b854"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:03:56.133730 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.133710    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c98f0c5-7aee-4ed2-989a-93547b78b03f-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:03:56.133800 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.133731    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/039b2915-0615-4c35-b18c-ca9ec6a7b854-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:03:56.135342 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.135314    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c98f0c5-7aee-4ed2-989a-93547b78b03f-kube-api-access-s6rhw" (OuterVolumeSpecName: "kube-api-access-s6rhw") pod "9c98f0c5-7aee-4ed2-989a-93547b78b03f" (UID: "9c98f0c5-7aee-4ed2-989a-93547b78b03f"). InnerVolumeSpecName "kube-api-access-s6rhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:03:56.135783 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.135756    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039b2915-0615-4c35-b18c-ca9ec6a7b854-kube-api-access-jfvdv" (OuterVolumeSpecName: "kube-api-access-jfvdv") pod "039b2915-0615-4c35-b18c-ca9ec6a7b854" (UID: "039b2915-0615-4c35-b18c-ca9ec6a7b854"). InnerVolumeSpecName "kube-api-access-jfvdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:03:56.234925 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.234886    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfvdv\" (UniqueName: \"kubernetes.io/projected/039b2915-0615-4c35-b18c-ca9ec6a7b854-kube-api-access-jfvdv\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:03:56.234925 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.234928    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6rhw\" (UniqueName: \"kubernetes.io/projected/9c98f0c5-7aee-4ed2-989a-93547b78b03f-kube-api-access-s6rhw\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:03:56.274673 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.274598    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:56.375981 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.375945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:56.798707 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.798671    2577 generic.go:356] "Generic (PLEG): container finished" podID="9c98f0c5-7aee-4ed2-989a-93547b78b03f" containerID="41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89" exitCode=0
Sep 16 17:03:56.799121 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.798738    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"
Sep 16 17:03:56.799121 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.798765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p" event={"ID":"9c98f0c5-7aee-4ed2-989a-93547b78b03f","Type":"ContainerDied","Data":"41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89"}
Sep 16 17:03:56.799121 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.798822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p" event={"ID":"9c98f0c5-7aee-4ed2-989a-93547b78b03f","Type":"ContainerDied","Data":"1b88a7a89ee6a57048971f278f002a71505f226ee5b82a20e52f102067414af5"}
Sep 16 17:03:56.799121 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.798843    2577 scope.go:117] "RemoveContainer" containerID="41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89"
Sep 16 17:03:56.800338 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.800314    2577 generic.go:356] "Generic (PLEG): container finished" podID="039b2915-0615-4c35-b18c-ca9ec6a7b854" containerID="8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127" exitCode=0
Sep 16 17:03:56.800426 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.800357    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"
Sep 16 17:03:56.800426 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.800401    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv" event={"ID":"039b2915-0615-4c35-b18c-ca9ec6a7b854","Type":"ContainerDied","Data":"8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127"}
Sep 16 17:03:56.800572 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.800452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv" event={"ID":"039b2915-0615-4c35-b18c-ca9ec6a7b854","Type":"ContainerDied","Data":"2c3acfb64b0722089f74918d748b6c67452222636a71aff71cb7dc5bcde843d6"}
Sep 16 17:03:56.810039 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.810020    2577 scope.go:117] "RemoveContainer" containerID="41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89"
Sep 16 17:03:56.810300 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:56.810282    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89\": container with ID starting with 41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89 not found: ID does not exist" containerID="41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89"
Sep 16 17:03:56.810367 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.810307    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89"} err="failed to get container status \"41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89\": rpc error: code = NotFound desc = could not find container \"41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89\": container with ID starting with 41d7d39803a0df6e23c6f7ce6918e540160c33faff068a8958324301020a1e89 not found: ID does not exist"
Sep 16 17:03:56.810367 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.810321    2577 scope.go:117] "RemoveContainer" containerID="8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127"
Sep 16 17:03:56.818838 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.818818    2577 scope.go:117] "RemoveContainer" containerID="8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127"
Sep 16 17:03:56.819123 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:03:56.819106    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127\": container with ID starting with 8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127 not found: ID does not exist" containerID="8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127"
Sep 16 17:03:56.819173 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.819128    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127"} err="failed to get container status \"8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127\": rpc error: code = NotFound desc = could not find container \"8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127\": container with ID starting with 8cb2d6dd929f989e88f433e4d461ef161e993ea15043914b1b5bc0c29dac7127 not found: ID does not exist"
Sep 16 17:03:56.832598 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.832574    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"]
Sep 16 17:03:56.837469 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.837446    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9359a3-58cd9bcdb6-l4x6p"]
Sep 16 17:03:56.849953 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.849933    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"]
Sep 16 17:03:56.855794 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.855766    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9359a3-759cc6947f-bhfdv"]
Sep 16 17:03:56.977872 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.977842    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="039b2915-0615-4c35-b18c-ca9ec6a7b854" path="/var/lib/kubelet/pods/039b2915-0615-4c35-b18c-ca9ec6a7b854/volumes"
Sep 16 17:03:56.978192 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.978179    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c98f0c5-7aee-4ed2-989a-93547b78b03f" path="/var/lib/kubelet/pods/9c98f0c5-7aee-4ed2-989a-93547b78b03f/volumes"
Sep 16 17:03:56.978474 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:56.978463    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afb8d763-ebbc-44c9-9422-2fc89385a42b" path="/var/lib/kubelet/pods/afb8d763-ebbc-44c9-9422-2fc89385a42b/volumes"
Sep 16 17:03:57.260733 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:57.260700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:58.039253 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:58.039218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:58.900568 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:58.900500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:03:59.617687 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:03:59.617638    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:00.322244 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:00.322203    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:02.119223 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.119190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:02.219482 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.219430    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:02.375518 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.375439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:02.705864 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.705827    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"]
Sep 16 17:04:02.706562 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706515    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c98f0c5-7aee-4ed2-989a-93547b78b03f" containerName="payload-pre-processing"
Sep 16 17:04:02.706562 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706559    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c98f0c5-7aee-4ed2-989a-93547b78b03f" containerName="payload-pre-processing"
Sep 16 17:04:02.706740 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706609    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afb8d763-ebbc-44c9-9422-2fc89385a42b" containerName="revoke-keys"
Sep 16 17:04:02.706740 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706618    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="afb8d763-ebbc-44c9-9422-2fc89385a42b" containerName="revoke-keys"
Sep 16 17:04:02.706740 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706634    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="039b2915-0615-4c35-b18c-ca9ec6a7b854" containerName="payload-processing"
Sep 16 17:04:02.706740 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706645    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="039b2915-0615-4c35-b18c-ca9ec6a7b854" containerName="payload-processing"
Sep 16 17:04:02.706887 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706777    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="afb8d763-ebbc-44c9-9422-2fc89385a42b" containerName="revoke-keys"
Sep 16 17:04:02.706887 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706791    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="039b2915-0615-4c35-b18c-ca9ec6a7b854" containerName="payload-processing"
Sep 16 17:04:02.706887 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.706808    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c98f0c5-7aee-4ed2-989a-93547b78b03f" containerName="payload-pre-processing"
Sep 16 17:04:02.711878 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.711856    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"
Sep 16 17:04:02.715183 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.715157    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:04:02.716297 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.716273    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:04:02.719819 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.719795    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"]
Sep 16 17:04:02.894577 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.894521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cfc22c-a4ab-4e98-8517-6a2e2302eb3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq\" (UID: \"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"
Sep 16 17:04:02.894776 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.894736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:02.995651 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.995570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cfc22c-a4ab-4e98-8517-6a2e2302eb3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq\" (UID: \"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"
Sep 16 17:04:02.996199 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:02.996181    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cfc22c-a4ab-4e98-8517-6a2e2302eb3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq\" (UID: \"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"
Sep 16 17:04:03.025502 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:03.025481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"
Sep 16 17:04:03.301751 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:03.301724    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"]
Sep 16 17:04:03.302692 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:04:03.302668    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12cfc22c_a4ab_4e98_8517_6a2e2302eb3b.slice/crio-fc482697560ae05c245c786455c1063dd8171bdedaf5444e57fb128ce9efbce8 WatchSource:0}: Error finding container fc482697560ae05c245c786455c1063dd8171bdedaf5444e57fb128ce9efbce8: Status 404 returned error can't find the container with id fc482697560ae05c245c786455c1063dd8171bdedaf5444e57fb128ce9efbce8
Sep 16 17:04:03.833841 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:03.833806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:03.845624 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:03.845587    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq" event={"ID":"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b","Type":"ContainerStarted","Data":"3470ead5537f8fb893e5f1ff5acb2deed327f0cb55c018977d863b1d828ab4c0"}
Sep 16 17:04:03.845772 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:03.845680    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq" event={"ID":"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b","Type":"ContainerStarted","Data":"fc482697560ae05c245c786455c1063dd8171bdedaf5444e57fb128ce9efbce8"}
Sep 16 17:04:03.863144 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:03.863058    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq" podStartSLOduration=1.86304427 podStartE2EDuration="1.86304427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:04:03.861842681 +0000 UTC m=+1659.358715797" watchObservedRunningTime="2026-09-16 17:04:03.86304427 +0000 UTC m=+1659.359917399"
Sep 16 17:04:04.696623 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:04.696579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:04.854307 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:04.854276    2577 generic.go:356] "Generic (PLEG): container finished" podID="12cfc22c-a4ab-4e98-8517-6a2e2302eb3b" containerID="3470ead5537f8fb893e5f1ff5acb2deed327f0cb55c018977d863b1d828ab4c0" exitCode=0
Sep 16 17:04:04.854504 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:04.854377    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq" event={"ID":"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b","Type":"ContainerDied","Data":"3470ead5537f8fb893e5f1ff5acb2deed327f0cb55c018977d863b1d828ab4c0"}
Sep 16 17:04:05.008213 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:05.008124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:05.439380 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:05.439341    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:05.997125 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:05.997100    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"
Sep 16 17:04:06.123228 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:06.123136    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cfc22c-a4ab-4e98-8517-6a2e2302eb3b-maas-api-service-ca\") pod \"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b\" (UID: \"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b\") "
Sep 16 17:04:06.123502 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:06.123479    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12cfc22c-a4ab-4e98-8517-6a2e2302eb3b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12cfc22c-a4ab-4e98-8517-6a2e2302eb3b" (UID: "12cfc22c-a4ab-4e98-8517-6a2e2302eb3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:06.224481 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:06.224448    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12cfc22c-a4ab-4e98-8517-6a2e2302eb3b-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:06.390916 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:06.390889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:06.868147 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:06.868067    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"
Sep 16 17:04:06.868147 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:06.868103    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq" event={"ID":"12cfc22c-a4ab-4e98-8517-6a2e2302eb3b","Type":"ContainerDied","Data":"fc482697560ae05c245c786455c1063dd8171bdedaf5444e57fb128ce9efbce8"}
Sep 16 17:04:06.868330 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:06.868149    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc482697560ae05c245c786455c1063dd8171bdedaf5444e57fb128ce9efbce8"
Sep 16 17:04:07.080594 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:07.080521    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:07.948321 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:07.948281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:18.330973 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.330929    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:18.360792 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.360760    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"]
Sep 16 17:04:18.361034 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.361008    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4" podUID="fd7a9301-cd66-413a-abf1-f3c52019a058" containerName="payload-processing" containerID="cri-o://8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75" gracePeriod=30
Sep 16 17:04:18.364804 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.364766    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"]
Sep 16 17:04:18.365058 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.365033    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5" podUID="35f0869e-367c-4c6c-a185-f9baaf3748e0" containerName="payload-pre-processing" containerID="cri-o://e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7" gracePeriod=30
Sep 16 17:04:18.407619 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.407584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:18.507188 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.507117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:18.557538 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.557495    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"]
Sep 16 17:04:18.561897 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.561871    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-4acea1-jlwsq"]
Sep 16 17:04:18.596084 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.595998    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:18.640484 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.640453    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:04:18.647704 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.647685    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:04:18.737935 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.737904    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w4hw\" (UniqueName: \"kubernetes.io/projected/fd7a9301-cd66-413a-abf1-f3c52019a058-kube-api-access-6w4hw\") pod \"fd7a9301-cd66-413a-abf1-f3c52019a058\" (UID: \"fd7a9301-cd66-413a-abf1-f3c52019a058\") "
Sep 16 17:04:18.738099 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.737944    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd7a9301-cd66-413a-abf1-f3c52019a058-plugins-config-volume\") pod \"fd7a9301-cd66-413a-abf1-f3c52019a058\" (UID: \"fd7a9301-cd66-413a-abf1-f3c52019a058\") "
Sep 16 17:04:18.738099 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.738069    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0869e-367c-4c6c-a185-f9baaf3748e0-plugins-config-volume\") pod \"35f0869e-367c-4c6c-a185-f9baaf3748e0\" (UID: \"35f0869e-367c-4c6c-a185-f9baaf3748e0\") "
Sep 16 17:04:18.738180 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.738120    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47fdt\" (UniqueName: \"kubernetes.io/projected/35f0869e-367c-4c6c-a185-f9baaf3748e0-kube-api-access-47fdt\") pod \"35f0869e-367c-4c6c-a185-f9baaf3748e0\" (UID: \"35f0869e-367c-4c6c-a185-f9baaf3748e0\") "
Sep 16 17:04:18.738416 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.738392    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd7a9301-cd66-413a-abf1-f3c52019a058-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd7a9301-cd66-413a-abf1-f3c52019a058" (UID: "fd7a9301-cd66-413a-abf1-f3c52019a058"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:18.738477 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.738455    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35f0869e-367c-4c6c-a185-f9baaf3748e0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35f0869e-367c-4c6c-a185-f9baaf3748e0" (UID: "35f0869e-367c-4c6c-a185-f9baaf3748e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:18.740061 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.740041    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd7a9301-cd66-413a-abf1-f3c52019a058-kube-api-access-6w4hw" (OuterVolumeSpecName: "kube-api-access-6w4hw") pod "fd7a9301-cd66-413a-abf1-f3c52019a058" (UID: "fd7a9301-cd66-413a-abf1-f3c52019a058"). InnerVolumeSpecName "kube-api-access-6w4hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:18.740120 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.740105    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35f0869e-367c-4c6c-a185-f9baaf3748e0-kube-api-access-47fdt" (OuterVolumeSpecName: "kube-api-access-47fdt") pod "35f0869e-367c-4c6c-a185-f9baaf3748e0" (UID: "35f0869e-367c-4c6c-a185-f9baaf3748e0"). InnerVolumeSpecName "kube-api-access-47fdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:18.839852 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.839822    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35f0869e-367c-4c6c-a185-f9baaf3748e0-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:18.839852 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.839845    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47fdt\" (UniqueName: \"kubernetes.io/projected/35f0869e-367c-4c6c-a185-f9baaf3748e0-kube-api-access-47fdt\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:18.839852 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.839857    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w4hw\" (UniqueName: \"kubernetes.io/projected/fd7a9301-cd66-413a-abf1-f3c52019a058-kube-api-access-6w4hw\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:18.840074 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.839866    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd7a9301-cd66-413a-abf1-f3c52019a058-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:18.903900 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.903838    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:18.943090 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.943060    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd7a9301-cd66-413a-abf1-f3c52019a058" containerID="8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75" exitCode=0
Sep 16 17:04:18.943251 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.943108    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"
Sep 16 17:04:18.943251 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.943144    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4" event={"ID":"fd7a9301-cd66-413a-abf1-f3c52019a058","Type":"ContainerDied","Data":"8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75"}
Sep 16 17:04:18.943251 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.943203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4" event={"ID":"fd7a9301-cd66-413a-abf1-f3c52019a058","Type":"ContainerDied","Data":"ca300eba5f3b400e77940d6564de6520a60059f06dc361da179bfa7fe39457e3"}
Sep 16 17:04:18.943251 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.943232    2577 scope.go:117] "RemoveContainer" containerID="8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75"
Sep 16 17:04:18.944765 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.944741    2577 generic.go:356] "Generic (PLEG): container finished" podID="35f0869e-367c-4c6c-a185-f9baaf3748e0" containerID="e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7" exitCode=0
Sep 16 17:04:18.944897 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.944791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5" event={"ID":"35f0869e-367c-4c6c-a185-f9baaf3748e0","Type":"ContainerDied","Data":"e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7"}
Sep 16 17:04:18.944897 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.944804    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"
Sep 16 17:04:18.944897 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.944828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5" event={"ID":"35f0869e-367c-4c6c-a185-f9baaf3748e0","Type":"ContainerDied","Data":"0d4f8fe8c9e406c20e104292c8cc6a20407b34c9a8423b3a5e3f66ff7eaec6d1"}
Sep 16 17:04:18.953518 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.953500    2577 scope.go:117] "RemoveContainer" containerID="8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75"
Sep 16 17:04:18.953835 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:04:18.953814    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75\": container with ID starting with 8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75 not found: ID does not exist" containerID="8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75"
Sep 16 17:04:18.953932 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.953845    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75"} err="failed to get container status \"8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75\": rpc error: code = NotFound desc = could not find container \"8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75\": container with ID starting with 8f03f44db7ccc5c5219a1b7b90964ab9bd9cc2915ef19cd6e78e39eaebaf1f75 not found: ID does not exist"
Sep 16 17:04:18.953932 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.953864    2577 scope.go:117] "RemoveContainer" containerID="e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7"
Sep 16 17:04:18.962939 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.962920    2577 scope.go:117] "RemoveContainer" containerID="e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7"
Sep 16 17:04:18.963290 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:04:18.963239    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7\": container with ID starting with e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7 not found: ID does not exist" containerID="e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7"
Sep 16 17:04:18.963387 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.963298    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7"} err="failed to get container status \"e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7\": rpc error: code = NotFound desc = could not find container \"e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7\": container with ID starting with e5aa8618239b6c1b10cea5a42d7fc662e10daa9af307b6880b901990e616ccd7 not found: ID does not exist"
Sep 16 17:04:18.970826 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.970803    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"]
Sep 16 17:04:18.977821 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.977801    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12cfc22c-a4ab-4e98-8517-6a2e2302eb3b" path="/var/lib/kubelet/pods/12cfc22c-a4ab-4e98-8517-6a2e2302eb3b/volumes"
Sep 16 17:04:18.978077 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.978064    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4acea1-58789f44c5-skxb4"]
Sep 16 17:04:18.986902 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.986883    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"]
Sep 16 17:04:18.990441 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:18.990422    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-4acea1-56847c7df8-9prc5"]
Sep 16 17:04:19.006210 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:19.006187    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:19.767847 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:19.767810    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:20.878042 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:20.878006    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:20.977340 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:20.977305    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35f0869e-367c-4c6c-a185-f9baaf3748e0" path="/var/lib/kubelet/pods/35f0869e-367c-4c6c-a185-f9baaf3748e0/volumes"
Sep 16 17:04:20.977691 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:20.977677    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd7a9301-cd66-413a-abf1-f3c52019a058" path="/var/lib/kubelet/pods/fd7a9301-cd66-413a-abf1-f3c52019a058/volumes"
Sep 16 17:04:21.610447 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:21.610399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:22.331386 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:22.331344    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:24.826315 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:24.826279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:24.930153 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:24.930116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:25.085744 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.085649    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:25.312792 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.312763    2577 scope.go:117] "RemoveContainer" containerID="8c01af949fb7ee27475209a9b3f6a09065ba27a2d65dcc018f1ca0abccf75e3e"
Sep 16 17:04:25.403914 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.403875    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"]
Sep 16 17:04:25.404557 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404520    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd7a9301-cd66-413a-abf1-f3c52019a058" containerName="payload-processing"
Sep 16 17:04:25.404615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404560    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd7a9301-cd66-413a-abf1-f3c52019a058" containerName="payload-processing"
Sep 16 17:04:25.404615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404574    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35f0869e-367c-4c6c-a185-f9baaf3748e0" containerName="payload-pre-processing"
Sep 16 17:04:25.404615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404583    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35f0869e-367c-4c6c-a185-f9baaf3748e0" containerName="payload-pre-processing"
Sep 16 17:04:25.404708 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404622    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12cfc22c-a4ab-4e98-8517-6a2e2302eb3b" containerName="revoke-keys"
Sep 16 17:04:25.404708 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404631    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="12cfc22c-a4ab-4e98-8517-6a2e2302eb3b" containerName="revoke-keys"
Sep 16 17:04:25.404766 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404738    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd7a9301-cd66-413a-abf1-f3c52019a058" containerName="payload-processing"
Sep 16 17:04:25.404766 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404748    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="35f0869e-367c-4c6c-a185-f9baaf3748e0" containerName="payload-pre-processing"
Sep 16 17:04:25.404828 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.404766    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="12cfc22c-a4ab-4e98-8517-6a2e2302eb3b" containerName="revoke-keys"
Sep 16 17:04:25.409943 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.409921    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"
Sep 16 17:04:25.412978 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.412957    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:04:25.413263 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.413236    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:04:25.413988 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.413964    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"]
Sep 16 17:04:25.500563 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.500507    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9556b0b-bd99-4e1b-a973-87b8bac6191b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf\" (UID: \"a9556b0b-bd99-4e1b-a973-87b8bac6191b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"
Sep 16 17:04:25.601231 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.601160    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9556b0b-bd99-4e1b-a973-87b8bac6191b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf\" (UID: \"a9556b0b-bd99-4e1b-a973-87b8bac6191b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"
Sep 16 17:04:25.601839 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.601820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9556b0b-bd99-4e1b-a973-87b8bac6191b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf\" (UID: \"a9556b0b-bd99-4e1b-a973-87b8bac6191b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"
Sep 16 17:04:25.698772 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.698734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:25.722573 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.722543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"
Sep 16 17:04:25.998920 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:25.998895    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"]
Sep 16 17:04:26.000247 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:04:26.000220    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9556b0b_bd99_4e1b_a973_87b8bac6191b.slice/crio-249fe0709d304e1ef40308b85caa6b80fb2f41fc81bc7e3e5c60dccd0abf71c3 WatchSource:0}: Error finding container 249fe0709d304e1ef40308b85caa6b80fb2f41fc81bc7e3e5c60dccd0abf71c3: Status 404 returned error can't find the container with id 249fe0709d304e1ef40308b85caa6b80fb2f41fc81bc7e3e5c60dccd0abf71c3
Sep 16 17:04:26.662049 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:26.662016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:26.996408 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:26.996329    2577 generic.go:356] "Generic (PLEG): container finished" podID="a9556b0b-bd99-4e1b-a973-87b8bac6191b" containerID="776094a1b4cbfcce9a69a3185e60fcc39fd56e5a75a447633cddd7d9b2be6fc0" exitCode=0
Sep 16 17:04:26.996408 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:26.996393    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf" event={"ID":"a9556b0b-bd99-4e1b-a973-87b8bac6191b","Type":"ContainerDied","Data":"776094a1b4cbfcce9a69a3185e60fcc39fd56e5a75a447633cddd7d9b2be6fc0"}
Sep 16 17:04:26.996626 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:26.996427    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf" event={"ID":"a9556b0b-bd99-4e1b-a973-87b8bac6191b","Type":"ContainerStarted","Data":"249fe0709d304e1ef40308b85caa6b80fb2f41fc81bc7e3e5c60dccd0abf71c3"}
Sep 16 17:04:28.096930 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:28.096891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:28.136302 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:28.136281    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"
Sep 16 17:04:28.228978 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:28.228953    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9556b0b-bd99-4e1b-a973-87b8bac6191b-maas-api-service-ca\") pod \"a9556b0b-bd99-4e1b-a973-87b8bac6191b\" (UID: \"a9556b0b-bd99-4e1b-a973-87b8bac6191b\") "
Sep 16 17:04:28.229296 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:28.229275    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9556b0b-bd99-4e1b-a973-87b8bac6191b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9556b0b-bd99-4e1b-a973-87b8bac6191b" (UID: "a9556b0b-bd99-4e1b-a973-87b8bac6191b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:28.330651 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:28.330579    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9556b0b-bd99-4e1b-a973-87b8bac6191b-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:28.500918 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:28.500889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:29.008733 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:29.008706    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"
Sep 16 17:04:29.008866 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:29.008743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf" event={"ID":"a9556b0b-bd99-4e1b-a973-87b8bac6191b","Type":"ContainerDied","Data":"249fe0709d304e1ef40308b85caa6b80fb2f41fc81bc7e3e5c60dccd0abf71c3"}
Sep 16 17:04:29.008866 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:29.008787    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="249fe0709d304e1ef40308b85caa6b80fb2f41fc81bc7e3e5c60dccd0abf71c3"
Sep 16 17:04:29.386294 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:29.386199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:30.154471 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:30.154441    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:36.082827 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.082788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:36.084196 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.084171    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"]
Sep 16 17:04:36.086730 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.086708    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-f315c3-29826300-4spd2"]
Sep 16 17:04:36.114050 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.114020    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"]
Sep 16 17:04:36.114263 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.114225    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb" podUID="2b1c872a-b985-4fde-9027-e342a5b8aa90" containerName="payload-processing" containerID="cri-o://0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581" gracePeriod=30
Sep 16 17:04:36.126035 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.126008    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"]
Sep 16 17:04:36.126258 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.126204    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn" podUID="dbf1b1d9-287c-41c8-9375-f2c666c00637" containerName="payload-pre-processing" containerID="cri-o://9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483" gracePeriod=30
Sep 16 17:04:36.162536 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.162491    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:36.234175 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.234146    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:36.304451 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.304418    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"]
Sep 16 17:04:36.309267 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.309237    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-f315c3-brrhf"]
Sep 16 17:04:36.311964 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.311924    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:36.407967 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.407944    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 17:04:36.411029 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.411011    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 17:04:36.503633 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.503601    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9p9fn\" (UniqueName: \"kubernetes.io/projected/2b1c872a-b985-4fde-9027-e342a5b8aa90-kube-api-access-9p9fn\") pod \"2b1c872a-b985-4fde-9027-e342a5b8aa90\" (UID: \"2b1c872a-b985-4fde-9027-e342a5b8aa90\") "
Sep 16 17:04:36.503808 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.503644    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbf1b1d9-287c-41c8-9375-f2c666c00637-plugins-config-volume\") pod \"dbf1b1d9-287c-41c8-9375-f2c666c00637\" (UID: \"dbf1b1d9-287c-41c8-9375-f2c666c00637\") "
Sep 16 17:04:36.503808 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.503692    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1c872a-b985-4fde-9027-e342a5b8aa90-plugins-config-volume\") pod \"2b1c872a-b985-4fde-9027-e342a5b8aa90\" (UID: \"2b1c872a-b985-4fde-9027-e342a5b8aa90\") "
Sep 16 17:04:36.503808 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.503748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t45d\" (UniqueName: \"kubernetes.io/projected/dbf1b1d9-287c-41c8-9375-f2c666c00637-kube-api-access-9t45d\") pod \"dbf1b1d9-287c-41c8-9375-f2c666c00637\" (UID: \"dbf1b1d9-287c-41c8-9375-f2c666c00637\") "
Sep 16 17:04:36.504097 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.504067    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbf1b1d9-287c-41c8-9375-f2c666c00637-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbf1b1d9-287c-41c8-9375-f2c666c00637" (UID: "dbf1b1d9-287c-41c8-9375-f2c666c00637"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:36.504233 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.504126    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b1c872a-b985-4fde-9027-e342a5b8aa90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b1c872a-b985-4fde-9027-e342a5b8aa90" (UID: "2b1c872a-b985-4fde-9027-e342a5b8aa90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:36.505857 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.505824    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf1b1d9-287c-41c8-9375-f2c666c00637-kube-api-access-9t45d" (OuterVolumeSpecName: "kube-api-access-9t45d") pod "dbf1b1d9-287c-41c8-9375-f2c666c00637" (UID: "dbf1b1d9-287c-41c8-9375-f2c666c00637"). InnerVolumeSpecName "kube-api-access-9t45d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:36.505978 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.505868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b1c872a-b985-4fde-9027-e342a5b8aa90-kube-api-access-9p9fn" (OuterVolumeSpecName: "kube-api-access-9p9fn") pod "2b1c872a-b985-4fde-9027-e342a5b8aa90" (UID: "2b1c872a-b985-4fde-9027-e342a5b8aa90"). InnerVolumeSpecName "kube-api-access-9p9fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:36.605201 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.605157    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t45d\" (UniqueName: \"kubernetes.io/projected/dbf1b1d9-287c-41c8-9375-f2c666c00637-kube-api-access-9t45d\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:36.605201 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.605198    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p9fn\" (UniqueName: \"kubernetes.io/projected/2b1c872a-b985-4fde-9027-e342a5b8aa90-kube-api-access-9p9fn\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:36.605452 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.605214    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbf1b1d9-287c-41c8-9375-f2c666c00637-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:36.605452 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.605227    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b1c872a-b985-4fde-9027-e342a5b8aa90-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:36.978098 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.978060    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3" path="/var/lib/kubelet/pods/13ff2aa5-60ce-4f15-a55d-b0851d4cbeb3/volumes"
Sep 16 17:04:36.978451 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:36.978431    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9556b0b-bd99-4e1b-a973-87b8bac6191b" path="/var/lib/kubelet/pods/a9556b0b-bd99-4e1b-a973-87b8bac6191b/volumes"
Sep 16 17:04:37.058947 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.058909    2577 generic.go:356] "Generic (PLEG): container finished" podID="dbf1b1d9-287c-41c8-9375-f2c666c00637" containerID="9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483" exitCode=0
Sep 16 17:04:37.059118 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.058974    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"
Sep 16 17:04:37.059118 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.058993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn" event={"ID":"dbf1b1d9-287c-41c8-9375-f2c666c00637","Type":"ContainerDied","Data":"9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483"}
Sep 16 17:04:37.059118 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.059052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn" event={"ID":"dbf1b1d9-287c-41c8-9375-f2c666c00637","Type":"ContainerDied","Data":"dd84bbe4414c0c88939162cd97d54b845e5bf8e06ec94c77f3598f86c65072bf"}
Sep 16 17:04:37.059118 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.059074    2577 scope.go:117] "RemoveContainer" containerID="9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483"
Sep 16 17:04:37.060475 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.060446    2577 generic.go:356] "Generic (PLEG): container finished" podID="2b1c872a-b985-4fde-9027-e342a5b8aa90" containerID="0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581" exitCode=0
Sep 16 17:04:37.060609 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.060476    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb" event={"ID":"2b1c872a-b985-4fde-9027-e342a5b8aa90","Type":"ContainerDied","Data":"0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581"}
Sep 16 17:04:37.060609 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.060518    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"
Sep 16 17:04:37.060722 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.060521    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb" event={"ID":"2b1c872a-b985-4fde-9027-e342a5b8aa90","Type":"ContainerDied","Data":"c6aa269cf2806be19c1a0614bbbf2755f9081d05b797bcd95e4077b52a0d025d"}
Sep 16 17:04:37.068367 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.068350    2577 scope.go:117] "RemoveContainer" containerID="9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483"
Sep 16 17:04:37.068660 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:04:37.068640    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483\": container with ID starting with 9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483 not found: ID does not exist" containerID="9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483"
Sep 16 17:04:37.068750 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.068670    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483"} err="failed to get container status \"9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483\": rpc error: code = NotFound desc = could not find container \"9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483\": container with ID starting with 9e78746387574c9fc81976193ef28b476114bc601cccb3f3f60e3e55f5962483 not found: ID does not exist"
Sep 16 17:04:37.068750 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.068692    2577 scope.go:117] "RemoveContainer" containerID="0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581"
Sep 16 17:04:37.076679 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.076657    2577 scope.go:117] "RemoveContainer" containerID="0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581"
Sep 16 17:04:37.076925 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:04:37.076909    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581\": container with ID starting with 0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581 not found: ID does not exist" containerID="0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581"
Sep 16 17:04:37.076973 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.076932    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581"} err="failed to get container status \"0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581\": rpc error: code = NotFound desc = could not find container \"0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581\": container with ID starting with 0e9bd061a8169ff078a8fe0bd1a4611562e56ab48162cd2dafd75b7c4ef55581 not found: ID does not exist"
Sep 16 17:04:37.081927 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.081905    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"]
Sep 16 17:04:37.092970 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.092951    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-f315c3-5989cc795f-7s6nb"]
Sep 16 17:04:37.131990 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.131964    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"]
Sep 16 17:04:37.150817 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.150791    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f315c3-cb9cb5f58-2lncn"]
Sep 16 17:04:37.185120 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.185087    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:37.352754 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.352672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:37.821868 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:37.821829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:38.553265 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:38.553226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:38.978352 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:38.978308    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b1c872a-b985-4fde-9027-e342a5b8aa90" path="/var/lib/kubelet/pods/2b1c872a-b985-4fde-9027-e342a5b8aa90/volumes"
Sep 16 17:04:38.978774 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:38.978755    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbf1b1d9-287c-41c8-9375-f2c666c00637" path="/var/lib/kubelet/pods/dbf1b1d9-287c-41c8-9375-f2c666c00637/volumes"
Sep 16 17:04:39.424853 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:39.424821    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:43.029290 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.029257    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:43.112988 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.112955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:43.297705 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.297620    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:43.626436 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626356    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"]
Sep 16 17:04:43.626764 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626750    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9556b0b-bd99-4e1b-a973-87b8bac6191b" containerName="revoke-keys"
Sep 16 17:04:43.626812 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626765    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9556b0b-bd99-4e1b-a973-87b8bac6191b" containerName="revoke-keys"
Sep 16 17:04:43.626812 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626789    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b1c872a-b985-4fde-9027-e342a5b8aa90" containerName="payload-processing"
Sep 16 17:04:43.626812 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626795    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b1c872a-b985-4fde-9027-e342a5b8aa90" containerName="payload-processing"
Sep 16 17:04:43.626812 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626802    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf1b1d9-287c-41c8-9375-f2c666c00637" containerName="payload-pre-processing"
Sep 16 17:04:43.626812 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626808    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf1b1d9-287c-41c8-9375-f2c666c00637" containerName="payload-pre-processing"
Sep 16 17:04:43.626965 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626871    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9556b0b-bd99-4e1b-a973-87b8bac6191b" containerName="revoke-keys"
Sep 16 17:04:43.626965 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626881    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbf1b1d9-287c-41c8-9375-f2c666c00637" containerName="payload-pre-processing"
Sep 16 17:04:43.626965 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.626889    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b1c872a-b985-4fde-9027-e342a5b8aa90" containerName="payload-processing"
Sep 16 17:04:43.631293 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.631275    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"
Sep 16 17:04:43.633787 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.633767    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:04:43.633860 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.633766    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:04:43.642787 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.642765    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"]
Sep 16 17:04:43.773546 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.773493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18ca925b-af17-4277-ba5d-e2e951a83155-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng\" (UID: \"18ca925b-af17-4277-ba5d-e2e951a83155\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"
Sep 16 17:04:43.874448 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.874396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18ca925b-af17-4277-ba5d-e2e951a83155-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng\" (UID: \"18ca925b-af17-4277-ba5d-e2e951a83155\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"
Sep 16 17:04:43.875065 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.875037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18ca925b-af17-4277-ba5d-e2e951a83155-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng\" (UID: \"18ca925b-af17-4277-ba5d-e2e951a83155\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"
Sep 16 17:04:43.943375 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.943342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"
Sep 16 17:04:43.994862 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:43.994765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:44.218747 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:44.218716    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"]
Sep 16 17:04:44.220016 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:04:44.219985    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18ca925b_af17_4277_ba5d_e2e951a83155.slice/crio-19106dde3ec534e5c49c24b154502eac016e9527cb3912d6f97f8a87df73efc3 WatchSource:0}: Error finding container 19106dde3ec534e5c49c24b154502eac016e9527cb3912d6f97f8a87df73efc3: Status 404 returned error can't find the container with id 19106dde3ec534e5c49c24b154502eac016e9527cb3912d6f97f8a87df73efc3
Sep 16 17:04:45.089237 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:45.089198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:45.112823 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:45.112785    2577 generic.go:356] "Generic (PLEG): container finished" podID="18ca925b-af17-4277-ba5d-e2e951a83155" containerID="3f1c48b13c2c7de021b083aa764df7d7bb87af0ea6eba698e784ccd1314c5ac2" exitCode=0
Sep 16 17:04:45.112990 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:45.112869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng" event={"ID":"18ca925b-af17-4277-ba5d-e2e951a83155","Type":"ContainerDied","Data":"3f1c48b13c2c7de021b083aa764df7d7bb87af0ea6eba698e784ccd1314c5ac2"}
Sep 16 17:04:45.112990 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:45.112922    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng" event={"ID":"18ca925b-af17-4277-ba5d-e2e951a83155","Type":"ContainerStarted","Data":"19106dde3ec534e5c49c24b154502eac016e9527cb3912d6f97f8a87df73efc3"}
Sep 16 17:04:46.013956 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:46.013920    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:46.256154 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:46.256123    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"
Sep 16 17:04:46.399827 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:46.399795    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18ca925b-af17-4277-ba5d-e2e951a83155-maas-api-service-ca\") pod \"18ca925b-af17-4277-ba5d-e2e951a83155\" (UID: \"18ca925b-af17-4277-ba5d-e2e951a83155\") "
Sep 16 17:04:46.400133 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:46.400111    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18ca925b-af17-4277-ba5d-e2e951a83155-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18ca925b-af17-4277-ba5d-e2e951a83155" (UID: "18ca925b-af17-4277-ba5d-e2e951a83155"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:46.501405 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:46.501375    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18ca925b-af17-4277-ba5d-e2e951a83155-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:46.758437 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:46.758348    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:47.128557 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:47.128462    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"
Sep 16 17:04:47.128557 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:47.128488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng" event={"ID":"18ca925b-af17-4277-ba5d-e2e951a83155","Type":"ContainerDied","Data":"19106dde3ec534e5c49c24b154502eac016e9527cb3912d6f97f8a87df73efc3"}
Sep 16 17:04:47.128557 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:47.128554    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19106dde3ec534e5c49c24b154502eac016e9527cb3912d6f97f8a87df73efc3"
Sep 16 17:04:47.681171 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:47.681135    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:54.239757 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.239703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:54.240159 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.239800    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"]
Sep 16 17:04:54.242403 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.242382    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-2ce8bc-29826300-pvjlx"]
Sep 16 17:04:54.268035 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.268003    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"]
Sep 16 17:04:54.268347 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.268317    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd" podUID="95d64df8-9242-40d0-b9bb-0ca5229d4fed" containerName="payload-processing" containerID="cri-o://ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38" gracePeriod=30
Sep 16 17:04:54.338707 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.338667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:54.466021 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.465979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:54.521992 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.521966    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 17:04:54.537437 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.537408    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"]
Sep 16 17:04:54.546894 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.546869    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-2ce8bc-qnrng"]
Sep 16 17:04:54.581700 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.581674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s9ds\" (UniqueName: \"kubernetes.io/projected/95d64df8-9242-40d0-b9bb-0ca5229d4fed-kube-api-access-7s9ds\") pod \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\" (UID: \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\") "
Sep 16 17:04:54.581827 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.581721    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d64df8-9242-40d0-b9bb-0ca5229d4fed-plugins-config-volume\") pod \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\" (UID: \"95d64df8-9242-40d0-b9bb-0ca5229d4fed\") "
Sep 16 17:04:54.582119 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.582095    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95d64df8-9242-40d0-b9bb-0ca5229d4fed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95d64df8-9242-40d0-b9bb-0ca5229d4fed" (UID: "95d64df8-9242-40d0-b9bb-0ca5229d4fed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:04:54.583845 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.583816    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95d64df8-9242-40d0-b9bb-0ca5229d4fed-kube-api-access-7s9ds" (OuterVolumeSpecName: "kube-api-access-7s9ds") pod "95d64df8-9242-40d0-b9bb-0ca5229d4fed" (UID: "95d64df8-9242-40d0-b9bb-0ca5229d4fed"). InnerVolumeSpecName "kube-api-access-7s9ds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:04:54.625475 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.625437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:54.683060 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.683023    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s9ds\" (UniqueName: \"kubernetes.io/projected/95d64df8-9242-40d0-b9bb-0ca5229d4fed-kube-api-access-7s9ds\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:54.683060 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.683049    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95d64df8-9242-40d0-b9bb-0ca5229d4fed-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:04:54.874763 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.874678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:54.978630 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.978597    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18ca925b-af17-4277-ba5d-e2e951a83155" path="/var/lib/kubelet/pods/18ca925b-af17-4277-ba5d-e2e951a83155/volumes"
Sep 16 17:04:54.978961 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.978945    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9af77317-6884-4c84-a4a2-5b7a42195060" path="/var/lib/kubelet/pods/9af77317-6884-4c84-a4a2-5b7a42195060/volumes"
Sep 16 17:04:54.983254 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:54.983228    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:55.178597 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.178562    2577 generic.go:356] "Generic (PLEG): container finished" podID="95d64df8-9242-40d0-b9bb-0ca5229d4fed" containerID="ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38" exitCode=0
Sep 16 17:04:55.178768 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.178604    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"
Sep 16 17:04:55.178768 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.178640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd" event={"ID":"95d64df8-9242-40d0-b9bb-0ca5229d4fed","Type":"ContainerDied","Data":"ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38"}
Sep 16 17:04:55.178768 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.178699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd" event={"ID":"95d64df8-9242-40d0-b9bb-0ca5229d4fed","Type":"ContainerDied","Data":"2062a6d16d24b5e8c5a7fa2b4fcf7f06a13c43f62b9322aeaa65dc7b0c033c49"}
Sep 16 17:04:55.178768 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.178726    2577 scope.go:117] "RemoveContainer" containerID="ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38"
Sep 16 17:04:55.188193 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.188171    2577 scope.go:117] "RemoveContainer" containerID="ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38"
Sep 16 17:04:55.188471 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:04:55.188452    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38\": container with ID starting with ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38 not found: ID does not exist" containerID="ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38"
Sep 16 17:04:55.188541 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.188478    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38"} err="failed to get container status \"ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38\": rpc error: code = NotFound desc = could not find container \"ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38\": container with ID starting with ac0e52f7e1f47eb1c7fe15f8dd6335b49d05e0448ec3dbd40e6c10d3282e5a38 not found: ID does not exist"
Sep 16 17:04:55.203411 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.203385    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"]
Sep 16 17:04:55.211703 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.211671    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-2ce8bc-6cbc45685-f6jvd"]
Sep 16 17:04:55.847436 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:55.847395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:56.907646 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:56.907606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:56.977018 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:56.976985    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95d64df8-9242-40d0-b9bb-0ca5229d4fed" path="/var/lib/kubelet/pods/95d64df8-9242-40d0-b9bb-0ca5229d4fed/volumes"
Sep 16 17:04:57.202548 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:57.202433    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:58.228299 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:58.228264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:58.641907 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:58.641869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:58.960085 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:58.959995    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:04:59.245979 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:04:59.245889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:01.878577 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:01.878517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:01.978784 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:01.978745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:02.650942 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:02.650903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:02.754193 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:02.754096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:03.496086 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:03.496044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:04.213288 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:04.213253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:04.478977 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:04.478891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:04.577987 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:04.577943    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:04.941517 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:04.941470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:05.035566 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.035506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:05.066666 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.066633    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"]
Sep 16 17:05:05.067076 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.067058    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18ca925b-af17-4277-ba5d-e2e951a83155" containerName="revoke-keys"
Sep 16 17:05:05.067076 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.067075    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="18ca925b-af17-4277-ba5d-e2e951a83155" containerName="revoke-keys"
Sep 16 17:05:05.067206 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.067087    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d64df8-9242-40d0-b9bb-0ca5229d4fed" containerName="payload-processing"
Sep 16 17:05:05.067206 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.067093    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d64df8-9242-40d0-b9bb-0ca5229d4fed" containerName="payload-processing"
Sep 16 17:05:05.067206 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.067157    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95d64df8-9242-40d0-b9bb-0ca5229d4fed" containerName="payload-processing"
Sep 16 17:05:05.067206 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.067167    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="18ca925b-af17-4277-ba5d-e2e951a83155" containerName="revoke-keys"
Sep 16 17:05:05.070455 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.070424    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:05.072972 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.072952    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-9ac315\""
Sep 16 17:05:05.073309 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.073290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-9ac315-dockercfg-jf5cd\""
Sep 16 17:05:05.089058 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.089034    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"]
Sep 16 17:05:05.121313 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.121285    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"]
Sep 16 17:05:05.124754 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.124739    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:05.135724 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.135702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"]
Sep 16 17:05:05.179405 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.179372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/299d0d1c-7c7f-41f7-9103-76723f28715b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn\" (UID: \"299d0d1c-7c7f-41f7-9103-76723f28715b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:05.179405 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.179408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpqpg\" (UniqueName: \"kubernetes.io/projected/299d0d1c-7c7f-41f7-9103-76723f28715b-kube-api-access-bpqpg\") pod \"payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn\" (UID: \"299d0d1c-7c7f-41f7-9103-76723f28715b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:05.179621 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.179443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1875a44-4c8f-4d76-8f19-4c77314f9558-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj\" (UID: \"b1875a44-4c8f-4d76-8f19-4c77314f9558\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:05.179621 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.179538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-498zh\" (UniqueName: \"kubernetes.io/projected/b1875a44-4c8f-4d76-8f19-4c77314f9558-kube-api-access-498zh\") pod \"payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj\" (UID: \"b1875a44-4c8f-4d76-8f19-4c77314f9558\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:05.280970 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.280876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/299d0d1c-7c7f-41f7-9103-76723f28715b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn\" (UID: \"299d0d1c-7c7f-41f7-9103-76723f28715b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:05.280970 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.280924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpqpg\" (UniqueName: \"kubernetes.io/projected/299d0d1c-7c7f-41f7-9103-76723f28715b-kube-api-access-bpqpg\") pod \"payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn\" (UID: \"299d0d1c-7c7f-41f7-9103-76723f28715b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:05.280970 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.280951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1875a44-4c8f-4d76-8f19-4c77314f9558-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj\" (UID: \"b1875a44-4c8f-4d76-8f19-4c77314f9558\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:05.281264 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.280986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498zh\" (UniqueName: \"kubernetes.io/projected/b1875a44-4c8f-4d76-8f19-4c77314f9558-kube-api-access-498zh\") pod \"payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj\" (UID: \"b1875a44-4c8f-4d76-8f19-4c77314f9558\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:05.281645 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.281622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/299d0d1c-7c7f-41f7-9103-76723f28715b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn\" (UID: \"299d0d1c-7c7f-41f7-9103-76723f28715b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:05.281645 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.281635    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1875a44-4c8f-4d76-8f19-4c77314f9558-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj\" (UID: \"b1875a44-4c8f-4d76-8f19-4c77314f9558\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:05.289918 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.289885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpqpg\" (UniqueName: \"kubernetes.io/projected/299d0d1c-7c7f-41f7-9103-76723f28715b-kube-api-access-bpqpg\") pod \"payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn\" (UID: \"299d0d1c-7c7f-41f7-9103-76723f28715b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:05.290059 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.289934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-498zh\" (UniqueName: \"kubernetes.io/projected/b1875a44-4c8f-4d76-8f19-4c77314f9558-kube-api-access-498zh\") pod \"payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj\" (UID: \"b1875a44-4c8f-4d76-8f19-4c77314f9558\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:05.380347 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.380319    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:05.435837 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.435305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:05.601176 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.601106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:05.680015 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.679993    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"]
Sep 16 17:05:05.682192 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:05:05.682165    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1875a44_4c8f_4d76_8f19_4c77314f9558.slice/crio-88419cbb4750b1aa798bbdf22e19eea92345cd181b9fa2afe775feff7a95298c WatchSource:0}: Error finding container 88419cbb4750b1aa798bbdf22e19eea92345cd181b9fa2afe775feff7a95298c: Status 404 returned error can't find the container with id 88419cbb4750b1aa798bbdf22e19eea92345cd181b9fa2afe775feff7a95298c
Sep 16 17:05:05.723762 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:05.723737    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"]
Sep 16 17:05:05.726656 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:05:05.726632    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod299d0d1c_7c7f_41f7_9103_76723f28715b.slice/crio-25dcca1ea53cc7cef5f2e5435f0e835f1bad507d4d3507657af2da49051f475a WatchSource:0}: Error finding container 25dcca1ea53cc7cef5f2e5435f0e835f1bad507d4d3507657af2da49051f475a: Status 404 returned error can't find the container with id 25dcca1ea53cc7cef5f2e5435f0e835f1bad507d4d3507657af2da49051f475a
Sep 16 17:05:06.253962 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.253911    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj" event={"ID":"b1875a44-4c8f-4d76-8f19-4c77314f9558","Type":"ContainerStarted","Data":"268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771"}
Sep 16 17:05:06.254183 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.253982    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj" event={"ID":"b1875a44-4c8f-4d76-8f19-4c77314f9558","Type":"ContainerStarted","Data":"88419cbb4750b1aa798bbdf22e19eea92345cd181b9fa2afe775feff7a95298c"}
Sep 16 17:05:06.254183 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.254014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:06.255792 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.255760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn" event={"ID":"299d0d1c-7c7f-41f7-9103-76723f28715b","Type":"ContainerStarted","Data":"b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b"}
Sep 16 17:05:06.255908 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.255813    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn" event={"ID":"299d0d1c-7c7f-41f7-9103-76723f28715b","Type":"ContainerStarted","Data":"25dcca1ea53cc7cef5f2e5435f0e835f1bad507d4d3507657af2da49051f475a"}
Sep 16 17:05:06.255908 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.255842    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:06.274706 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.274668    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj" podStartSLOduration=1.2746539860000001 podStartE2EDuration="1.274653986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:05:06.272459643 +0000 UTC m=+1721.769332767" watchObservedRunningTime="2026-09-16 17:05:06.274653986 +0000 UTC m=+1721.771527164"
Sep 16 17:05:06.319160 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.319107    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn" podStartSLOduration=1.319087837 podStartE2EDuration="1.319087837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:05:06.318850428 +0000 UTC m=+1721.815723552" watchObservedRunningTime="2026-09-16 17:05:06.319087837 +0000 UTC m=+1721.815960962"
Sep 16 17:05:06.491420 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:06.491385    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:17.264157 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:17.264122    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:05:17.264733 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:17.264282    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:05:26.408724 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:26.408689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:27.168203 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:27.168147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:27.346776 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:27.346740    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:27.870119 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:27.870077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:28.010679 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:28.010644    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:28.336371 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:28.336325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:28.444246 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:28.444212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:28.783495 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:28.783448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:30.985669 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:30.985632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:31.568619 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:31.568578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:31.675159 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:31.675126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:31.848135 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:31.848034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:32.812810 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:32.812761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:33.743694 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:33.743655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:34.483399 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:34.483362    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:34.580587 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:34.580558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:34.969702 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:34.969668    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:35.064296 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.064258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:35.070308 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.070283    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"]
Sep 16 17:05:35.073590 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.073570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:35.076057 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.076038    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-6586c3\""
Sep 16 17:05:35.076164 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.076061    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-6586c3-dockercfg-hznn7\""
Sep 16 17:05:35.085118 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.085096    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"]
Sep 16 17:05:35.159852 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.159800    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2\" (UID: \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:35.159852 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.159852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg8sc\" (UniqueName: \"kubernetes.io/projected/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-kube-api-access-sg8sc\") pod \"payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2\" (UID: \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:35.261169 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.261074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2\" (UID: \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:35.261169 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.261119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg8sc\" (UniqueName: \"kubernetes.io/projected/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-kube-api-access-sg8sc\") pod \"payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2\" (UID: \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:35.261821 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.261800    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2\" (UID: \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:35.269457 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.269420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg8sc\" (UniqueName: \"kubernetes.io/projected/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-kube-api-access-sg8sc\") pod \"payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2\" (UID: \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:35.385224 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.385182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:35.662747 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:35.662646    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"]
Sep 16 17:05:35.663463 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:05:35.663437    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca3d81a_ad1b_4da2_b0a9_f237ef30b2fc.slice/crio-c0c14e00ec251c9e8b785bc8be6ce06e2596fad218c1de3d28f8e9fd51724a39 WatchSource:0}: Error finding container c0c14e00ec251c9e8b785bc8be6ce06e2596fad218c1de3d28f8e9fd51724a39: Status 404 returned error can't find the container with id c0c14e00ec251c9e8b785bc8be6ce06e2596fad218c1de3d28f8e9fd51724a39
Sep 16 17:05:36.438474 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:36.438439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2" event={"ID":"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc","Type":"ContainerStarted","Data":"981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e"}
Sep 16 17:05:36.438706 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:36.438503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2" event={"ID":"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc","Type":"ContainerStarted","Data":"c0c14e00ec251c9e8b785bc8be6ce06e2596fad218c1de3d28f8e9fd51724a39"}
Sep 16 17:05:36.438706 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:36.438671    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:36.456081 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:36.456031    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2" podStartSLOduration=1.456015063 podStartE2EDuration="1.456015063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:05:36.452957011 +0000 UTC m=+1751.949830147" watchObservedRunningTime="2026-09-16 17:05:36.456015063 +0000 UTC m=+1751.952888187"
Sep 16 17:05:47.447936 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:47.447902    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:05:57.470615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:57.470575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:58.274604 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:58.274560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:58.988585 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:58.988546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:05:59.781806 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:05:59.781771    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:00.662701 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:00.662668    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:01.636949 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:01.636915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:25.046616 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:25.046478    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:06:25.053213 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:25.053184    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:06:25.390825 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:25.390797    2577 scope.go:117] "RemoveContainer" containerID="aae0da164fce5611b635cf5aede79b5a4a9025e2a1a57ed08181a7735869ae60"
Sep 16 17:06:25.399877 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:25.399858    2577 scope.go:117] "RemoveContainer" containerID="b842bec0447b0c3a70c634fddc94e0d74966258ae39d76463434fa3b2b98f8e0"
Sep 16 17:06:25.408425 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:25.408404    2577 scope.go:117] "RemoveContainer" containerID="842ba3405e8e1b2f0d6eb3311054e30651e8994836b5fb9bd2a102e7d2b69e6d"
Sep 16 17:06:25.417439 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:25.417418    2577 scope.go:117] "RemoveContainer" containerID="1501fe5a3a0a895416babcb066a5b79cfa8e414ad9fbb164b6dd9f23695aa668"
Sep 16 17:06:26.306867 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:26.306828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:52.635964 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:52.635902    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj_b1875a44-4c8f-4d76-8f19-4c77314f9558/payload-processing/0.log"
Sep 16 17:06:52.964214 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:52.964180    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:54.399804 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:54.399768    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:55.301195 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:55.301157    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:56.060799 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:56.060764    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:56.179152 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:56.179118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:56.272956 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:56.272918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:57.119719 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:57.119680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:58.083727 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:58.083691    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:58.806593 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:58.806551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:59.604233 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:59.604196    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:59.681036 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:59.680997    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:06:59.782135 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:06:59.782100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:00.224172 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.224136    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:00.332514 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.332479    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:00.355169 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.355140    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"]
Sep 16 17:07:00.358918 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.358902    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:00.363268 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.363251    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-48fbf0\""
Sep 16 17:07:00.382822 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.382796    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"]
Sep 16 17:07:00.494240 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.494150    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"]
Sep 16 17:07:00.497822 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.497799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:00.502803 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.502782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-48fbf0-dockercfg-sjng7\""
Sep 16 17:07:00.507554 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.507509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c4031f-1e2a-4691-bec8-7648127f5be4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq\" (UID: \"88c4031f-1e2a-4691-bec8-7648127f5be4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:00.507640 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.507586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xdrg\" (UniqueName: \"kubernetes.io/projected/88c4031f-1e2a-4691-bec8-7648127f5be4-kube-api-access-2xdrg\") pod \"payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq\" (UID: \"88c4031f-1e2a-4691-bec8-7648127f5be4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:00.521938 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.521913    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"]
Sep 16 17:07:00.608364 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.608328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c4031f-1e2a-4691-bec8-7648127f5be4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq\" (UID: \"88c4031f-1e2a-4691-bec8-7648127f5be4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:00.608364 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.608369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a8cd77-755a-4f87-8a6f-87a5911f64d4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr\" (UID: \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:00.608597 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.608392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xdrg\" (UniqueName: \"kubernetes.io/projected/88c4031f-1e2a-4691-bec8-7648127f5be4-kube-api-access-2xdrg\") pod \"payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq\" (UID: \"88c4031f-1e2a-4691-bec8-7648127f5be4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:00.608597 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.608496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfztg\" (UniqueName: \"kubernetes.io/projected/38a8cd77-755a-4f87-8a6f-87a5911f64d4-kube-api-access-xfztg\") pod \"payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr\" (UID: \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:00.609175 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.609149    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c4031f-1e2a-4691-bec8-7648127f5be4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq\" (UID: \"88c4031f-1e2a-4691-bec8-7648127f5be4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:00.617643 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.617623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xdrg\" (UniqueName: \"kubernetes.io/projected/88c4031f-1e2a-4691-bec8-7648127f5be4-kube-api-access-2xdrg\") pod \"payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq\" (UID: \"88c4031f-1e2a-4691-bec8-7648127f5be4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:00.668710 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.668674    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:00.709577 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.709521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a8cd77-755a-4f87-8a6f-87a5911f64d4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr\" (UID: \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:00.709769 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.709653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfztg\" (UniqueName: \"kubernetes.io/projected/38a8cd77-755a-4f87-8a6f-87a5911f64d4-kube-api-access-xfztg\") pod \"payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr\" (UID: \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:00.710218 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.710195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a8cd77-755a-4f87-8a6f-87a5911f64d4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr\" (UID: \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:00.719381 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.719349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfztg\" (UniqueName: \"kubernetes.io/projected/38a8cd77-755a-4f87-8a6f-87a5911f64d4-kube-api-access-xfztg\") pod \"payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr\" (UID: \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:00.808408 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.808331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:00.992971 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:00.992940    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"]
Sep 16 17:07:00.993115 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:07:00.993021    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c4031f_1e2a_4691_bec8_7648127f5be4.slice/crio-5c68ee80f4bf14f192196c5f226d7dc600ec4832958e8d39cb533175bad2d2c9 WatchSource:0}: Error finding container 5c68ee80f4bf14f192196c5f226d7dc600ec4832958e8d39cb533175bad2d2c9: Status 404 returned error can't find the container with id 5c68ee80f4bf14f192196c5f226d7dc600ec4832958e8d39cb533175bad2d2c9
Sep 16 17:07:01.082639 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.082606    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"]
Sep 16 17:07:01.083845 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:07:01.083812    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a8cd77_755a_4f87_8a6f_87a5911f64d4.slice/crio-516430b78f93f939b614e6fa6311956c92b1ab6e1b3b22fc63e45b6770a060c6 WatchSource:0}: Error finding container 516430b78f93f939b614e6fa6311956c92b1ab6e1b3b22fc63e45b6770a060c6: Status 404 returned error can't find the container with id 516430b78f93f939b614e6fa6311956c92b1ab6e1b3b22fc63e45b6770a060c6
Sep 16 17:07:01.962912 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.962874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr" event={"ID":"38a8cd77-755a-4f87-8a6f-87a5911f64d4","Type":"ContainerStarted","Data":"0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923"}
Sep 16 17:07:01.963339 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.962940    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr" event={"ID":"38a8cd77-755a-4f87-8a6f-87a5911f64d4","Type":"ContainerStarted","Data":"516430b78f93f939b614e6fa6311956c92b1ab6e1b3b22fc63e45b6770a060c6"}
Sep 16 17:07:01.963339 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.962966    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:01.964625 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.964599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq" event={"ID":"88c4031f-1e2a-4691-bec8-7648127f5be4","Type":"ContainerStarted","Data":"1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1"}
Sep 16 17:07:01.964730 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.964643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq" event={"ID":"88c4031f-1e2a-4691-bec8-7648127f5be4","Type":"ContainerStarted","Data":"5c68ee80f4bf14f192196c5f226d7dc600ec4832958e8d39cb533175bad2d2c9"}
Sep 16 17:07:01.964730 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.964662    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:01.980689 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.980640    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr" podStartSLOduration=1.980626332 podStartE2EDuration="1.980626332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:07:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:07:01.97938411 +0000 UTC m=+1837.476257235" watchObservedRunningTime="2026-09-16 17:07:01.980626332 +0000 UTC m=+1837.477499457"
Sep 16 17:07:01.998538 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:01.998474    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq" podStartSLOduration=1.998456448 podStartE2EDuration="1.998456448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:07:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:07:01.995898747 +0000 UTC m=+1837.492771872" watchObservedRunningTime="2026-09-16 17:07:01.998456448 +0000 UTC m=+1837.495329574"
Sep 16 17:07:12.978030 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:12.977996    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:12.978435 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:12.978049    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:21.867190 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:21.867147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:22.684662 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:22.684602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:23.392201 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:23.392129    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:24.191578 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:24.191521    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:25.017615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:25.017583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:25.735508 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:25.735467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:26.033991 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.033902    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:26.669932 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.669893    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"]
Sep 16 17:07:26.674097 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.674075    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"
Sep 16 17:07:26.676830 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.676806    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:07:26.676950 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.676894    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:07:26.685765 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.685746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"]
Sep 16 17:07:26.746303 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.746271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77fe46d2-0f03-447f-b39a-97fdfa744996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw\" (UID: \"77fe46d2-0f03-447f-b39a-97fdfa744996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"
Sep 16 17:07:26.847450 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.847395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77fe46d2-0f03-447f-b39a-97fdfa744996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw\" (UID: \"77fe46d2-0f03-447f-b39a-97fdfa744996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"
Sep 16 17:07:26.848109 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.848085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77fe46d2-0f03-447f-b39a-97fdfa744996-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw\" (UID: \"77fe46d2-0f03-447f-b39a-97fdfa744996\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"
Sep 16 17:07:26.985564 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:26.985455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"
Sep 16 17:07:27.262656 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:27.262585    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"]
Sep 16 17:07:27.264111 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:07:27.264086    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77fe46d2_0f03_447f_b39a_97fdfa744996.slice/crio-a2173853528cff35f4c4f310e40e1250c10f3169c4849662a627d3aa4cc21135 WatchSource:0}: Error finding container a2173853528cff35f4c4f310e40e1250c10f3169c4849662a627d3aa4cc21135: Status 404 returned error can't find the container with id a2173853528cff35f4c4f310e40e1250c10f3169c4849662a627d3aa4cc21135
Sep 16 17:07:27.397649 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:27.397618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:28.097862 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:28.097829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:28.132215 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:28.132182    2577 generic.go:356] "Generic (PLEG): container finished" podID="77fe46d2-0f03-447f-b39a-97fdfa744996" containerID="11f4b096ff6d4a95817304ae6333cd0410726daf51992ee355caacbec166bc30" exitCode=0
Sep 16 17:07:28.132345 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:28.132266    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw" event={"ID":"77fe46d2-0f03-447f-b39a-97fdfa744996","Type":"ContainerDied","Data":"11f4b096ff6d4a95817304ae6333cd0410726daf51992ee355caacbec166bc30"}
Sep 16 17:07:28.132345 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:28.132318    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw" event={"ID":"77fe46d2-0f03-447f-b39a-97fdfa744996","Type":"ContainerStarted","Data":"a2173853528cff35f4c4f310e40e1250c10f3169c4849662a627d3aa4cc21135"}
Sep 16 17:07:28.806670 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:28.806627    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:29.302188 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:29.302164    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"
Sep 16 17:07:29.373061 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:29.373026    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77fe46d2-0f03-447f-b39a-97fdfa744996-maas-api-service-ca\") pod \"77fe46d2-0f03-447f-b39a-97fdfa744996\" (UID: \"77fe46d2-0f03-447f-b39a-97fdfa744996\") "
Sep 16 17:07:29.373390 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:29.373366    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77fe46d2-0f03-447f-b39a-97fdfa744996-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "77fe46d2-0f03-447f-b39a-97fdfa744996" (UID: "77fe46d2-0f03-447f-b39a-97fdfa744996"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:07:29.474499 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:29.474409    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/77fe46d2-0f03-447f-b39a-97fdfa744996-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:07:29.586255 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:29.586213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:30.149074 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:30.149045    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"
Sep 16 17:07:30.149514 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:30.149035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw" event={"ID":"77fe46d2-0f03-447f-b39a-97fdfa744996","Type":"ContainerDied","Data":"a2173853528cff35f4c4f310e40e1250c10f3169c4849662a627d3aa4cc21135"}
Sep 16 17:07:30.149514 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:30.149182    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2173853528cff35f4c4f310e40e1250c10f3169c4849662a627d3aa4cc21135"
Sep 16 17:07:30.754495 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:30.754440    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:31.206165 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:31.206105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:35.290065 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:35.290029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:37.854679 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:37.854622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:37.881904 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:37.881876    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"]
Sep 16 17:07:37.882102 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:37.882074    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr" podUID="38a8cd77-755a-4f87-8a6f-87a5911f64d4" containerName="payload-processing" containerID="cri-o://0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923" gracePeriod=30
Sep 16 17:07:37.893853 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:37.893816    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"]
Sep 16 17:07:37.894071 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:37.894039    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq" podUID="88c4031f-1e2a-4691-bec8-7648127f5be4" containerName="payload-pre-processing" containerID="cri-o://1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1" gracePeriod=30
Sep 16 17:07:37.922726 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:37.922694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:37.986109 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:37.986077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:38.050261 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.050190    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"]
Sep 16 17:07:38.051171 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.051047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:38.052921 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.052891    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-48fbf0-hnxnw"]
Sep 16 17:07:38.121857 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.121774    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:38.167108 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.167086    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:38.170237 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.170215    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:38.203590 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.203561    2577 generic.go:356] "Generic (PLEG): container finished" podID="88c4031f-1e2a-4691-bec8-7648127f5be4" containerID="1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1" exitCode=0
Sep 16 17:07:38.203750 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.203599    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"
Sep 16 17:07:38.203750 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.203618    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq" event={"ID":"88c4031f-1e2a-4691-bec8-7648127f5be4","Type":"ContainerDied","Data":"1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1"}
Sep 16 17:07:38.203750 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.203686    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq" event={"ID":"88c4031f-1e2a-4691-bec8-7648127f5be4","Type":"ContainerDied","Data":"5c68ee80f4bf14f192196c5f226d7dc600ec4832958e8d39cb533175bad2d2c9"}
Sep 16 17:07:38.203750 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.203724    2577 scope.go:117] "RemoveContainer" containerID="1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1"
Sep 16 17:07:38.206836 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.206811    2577 generic.go:356] "Generic (PLEG): container finished" podID="38a8cd77-755a-4f87-8a6f-87a5911f64d4" containerID="0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923" exitCode=0
Sep 16 17:07:38.207869 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.207610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr" event={"ID":"38a8cd77-755a-4f87-8a6f-87a5911f64d4","Type":"ContainerDied","Data":"0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923"}
Sep 16 17:07:38.207981 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.207926    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr" event={"ID":"38a8cd77-755a-4f87-8a6f-87a5911f64d4","Type":"ContainerDied","Data":"516430b78f93f939b614e6fa6311956c92b1ab6e1b3b22fc63e45b6770a060c6"}
Sep 16 17:07:38.207981 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.207648    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"
Sep 16 17:07:38.217777 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.217754    2577 scope.go:117] "RemoveContainer" containerID="1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1"
Sep 16 17:07:38.218095 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:07:38.218066    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1\": container with ID starting with 1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1 not found: ID does not exist" containerID="1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1"
Sep 16 17:07:38.218182 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.218103    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1"} err="failed to get container status \"1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1\": rpc error: code = NotFound desc = could not find container \"1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1\": container with ID starting with 1771253717dcf5b776c79e03819d30781f2c6f7baad4da279b5dc3e2e840a3c1 not found: ID does not exist"
Sep 16 17:07:38.218182 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.218120    2577 scope.go:117] "RemoveContainer" containerID="0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923"
Sep 16 17:07:38.228089 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.228064    2577 scope.go:117] "RemoveContainer" containerID="0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923"
Sep 16 17:07:38.228354 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:07:38.228337    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923\": container with ID starting with 0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923 not found: ID does not exist" containerID="0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923"
Sep 16 17:07:38.228410 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.228362    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923"} err="failed to get container status \"0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923\": rpc error: code = NotFound desc = could not find container \"0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923\": container with ID starting with 0c0a6d4b6237b0a705f394b0005f109143ba0082acab14a78e0eee1934ed8923 not found: ID does not exist"
Sep 16 17:07:38.252728 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.252707    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c4031f-1e2a-4691-bec8-7648127f5be4-plugins-config-volume\") pod \"88c4031f-1e2a-4691-bec8-7648127f5be4\" (UID: \"88c4031f-1e2a-4691-bec8-7648127f5be4\") "
Sep 16 17:07:38.252841 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.252747    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a8cd77-755a-4f87-8a6f-87a5911f64d4-plugins-config-volume\") pod \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\" (UID: \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\") "
Sep 16 17:07:38.252841 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.252778    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfztg\" (UniqueName: \"kubernetes.io/projected/38a8cd77-755a-4f87-8a6f-87a5911f64d4-kube-api-access-xfztg\") pod \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\" (UID: \"38a8cd77-755a-4f87-8a6f-87a5911f64d4\") "
Sep 16 17:07:38.252951 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.252868    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xdrg\" (UniqueName: \"kubernetes.io/projected/88c4031f-1e2a-4691-bec8-7648127f5be4-kube-api-access-2xdrg\") pod \"88c4031f-1e2a-4691-bec8-7648127f5be4\" (UID: \"88c4031f-1e2a-4691-bec8-7648127f5be4\") "
Sep 16 17:07:38.253101 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.253080    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38a8cd77-755a-4f87-8a6f-87a5911f64d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38a8cd77-755a-4f87-8a6f-87a5911f64d4" (UID: "38a8cd77-755a-4f87-8a6f-87a5911f64d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:07:38.253146 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.253091    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88c4031f-1e2a-4691-bec8-7648127f5be4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88c4031f-1e2a-4691-bec8-7648127f5be4" (UID: "88c4031f-1e2a-4691-bec8-7648127f5be4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:07:38.253190 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.253181    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c4031f-1e2a-4691-bec8-7648127f5be4-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:07:38.253226 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.253195    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a8cd77-755a-4f87-8a6f-87a5911f64d4-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:07:38.254745 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.254727    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38a8cd77-755a-4f87-8a6f-87a5911f64d4-kube-api-access-xfztg" (OuterVolumeSpecName: "kube-api-access-xfztg") pod "38a8cd77-755a-4f87-8a6f-87a5911f64d4" (UID: "38a8cd77-755a-4f87-8a6f-87a5911f64d4"). InnerVolumeSpecName "kube-api-access-xfztg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:07:38.254806 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.254772    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88c4031f-1e2a-4691-bec8-7648127f5be4-kube-api-access-2xdrg" (OuterVolumeSpecName: "kube-api-access-2xdrg") pod "88c4031f-1e2a-4691-bec8-7648127f5be4" (UID: "88c4031f-1e2a-4691-bec8-7648127f5be4"). InnerVolumeSpecName "kube-api-access-2xdrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:07:38.353848 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.353815    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xdrg\" (UniqueName: \"kubernetes.io/projected/88c4031f-1e2a-4691-bec8-7648127f5be4-kube-api-access-2xdrg\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:07:38.353848 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.353843    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfztg\" (UniqueName: \"kubernetes.io/projected/38a8cd77-755a-4f87-8a6f-87a5911f64d4-kube-api-access-xfztg\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:07:38.413106 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.413068    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:38.509283 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.509246    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:38.530112 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.530086    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"]
Sep 16 17:07:38.532676 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.532651    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-48fbf0-59bd976b8b-x4rpq"]
Sep 16 17:07:38.542032 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.542010    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"]
Sep 16 17:07:38.546656 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.546635    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-48fbf0-577db8bf9-cmnrr"]
Sep 16 17:07:38.977999 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.977962    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a8cd77-755a-4f87-8a6f-87a5911f64d4" path="/var/lib/kubelet/pods/38a8cd77-755a-4f87-8a6f-87a5911f64d4/volumes"
Sep 16 17:07:38.978366 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.978303    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77fe46d2-0f03-447f-b39a-97fdfa744996" path="/var/lib/kubelet/pods/77fe46d2-0f03-447f-b39a-97fdfa744996/volumes"
Sep 16 17:07:38.978634 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:38.978621    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88c4031f-1e2a-4691-bec8-7648127f5be4" path="/var/lib/kubelet/pods/88c4031f-1e2a-4691-bec8-7648127f5be4/volumes"
Sep 16 17:07:39.213645 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:39.213593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:40.081105 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:40.081057    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:41.017892 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:41.017857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:41.873742 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:41.873703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:44.168679 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:44.168641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:44.233789 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:44.233734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:44.424331 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:44.424224    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:47.849832 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.849740    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"]
Sep 16 17:07:47.850364 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850343    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a8cd77-755a-4f87-8a6f-87a5911f64d4" containerName="payload-processing"
Sep 16 17:07:47.850452 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850366    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a8cd77-755a-4f87-8a6f-87a5911f64d4" containerName="payload-processing"
Sep 16 17:07:47.850452 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850407    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88c4031f-1e2a-4691-bec8-7648127f5be4" containerName="payload-pre-processing"
Sep 16 17:07:47.850452 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850416    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88c4031f-1e2a-4691-bec8-7648127f5be4" containerName="payload-pre-processing"
Sep 16 17:07:47.850452 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850429    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77fe46d2-0f03-447f-b39a-97fdfa744996" containerName="revoke-keys"
Sep 16 17:07:47.850452 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850438    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="77fe46d2-0f03-447f-b39a-97fdfa744996" containerName="revoke-keys"
Sep 16 17:07:47.850732 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850555    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="77fe46d2-0f03-447f-b39a-97fdfa744996" containerName="revoke-keys"
Sep 16 17:07:47.850732 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850567    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38a8cd77-755a-4f87-8a6f-87a5911f64d4" containerName="payload-processing"
Sep 16 17:07:47.850732 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.850580    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88c4031f-1e2a-4691-bec8-7648127f5be4" containerName="payload-pre-processing"
Sep 16 17:07:47.855229 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.855209    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"
Sep 16 17:07:47.858268 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.858247    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:07:47.858379 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.858311    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:07:47.861036 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.861007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"]
Sep 16 17:07:47.940014 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:47.939977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e98444-307d-4f9f-b3a3-9550ef1b2eb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj\" (UID: \"91e98444-307d-4f9f-b3a3-9550ef1b2eb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"
Sep 16 17:07:48.041224 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:48.041181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e98444-307d-4f9f-b3a3-9550ef1b2eb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj\" (UID: \"91e98444-307d-4f9f-b3a3-9550ef1b2eb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"
Sep 16 17:07:48.041964 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:48.041941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e98444-307d-4f9f-b3a3-9550ef1b2eb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj\" (UID: \"91e98444-307d-4f9f-b3a3-9550ef1b2eb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"
Sep 16 17:07:48.150028 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:48.149996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:48.168279 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:48.168254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"
Sep 16 17:07:48.443117 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:48.443092    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"]
Sep 16 17:07:48.444818 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:07:48.444789    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e98444_307d_4f9f_b3a3_9550ef1b2eb7.slice/crio-8e755c2a37d04c6ef9805b4c94648bb9ab82e9327709033cd37124d60789e66c WatchSource:0}: Error finding container 8e755c2a37d04c6ef9805b4c94648bb9ab82e9327709033cd37124d60789e66c: Status 404 returned error can't find the container with id 8e755c2a37d04c6ef9805b4c94648bb9ab82e9327709033cd37124d60789e66c
Sep 16 17:07:48.980730 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:48.980634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:49.274624 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:49.274523    2577 generic.go:356] "Generic (PLEG): container finished" podID="91e98444-307d-4f9f-b3a3-9550ef1b2eb7" containerID="4816ce3aa38fd000b9ebd2666e7faa0c207cd7dead8648656c907c1853c76389" exitCode=0
Sep 16 17:07:49.274624 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:49.274590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj" event={"ID":"91e98444-307d-4f9f-b3a3-9550ef1b2eb7","Type":"ContainerDied","Data":"4816ce3aa38fd000b9ebd2666e7faa0c207cd7dead8648656c907c1853c76389"}
Sep 16 17:07:49.274808 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:49.274636    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj" event={"ID":"91e98444-307d-4f9f-b3a3-9550ef1b2eb7","Type":"ContainerStarted","Data":"8e755c2a37d04c6ef9805b4c94648bb9ab82e9327709033cd37124d60789e66c"}
Sep 16 17:07:49.787222 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:49.787176    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:50.421841 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:50.421817    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"
Sep 16 17:07:50.546376 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:50.546272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:50.566309 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:50.566282    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e98444-307d-4f9f-b3a3-9550ef1b2eb7-maas-api-service-ca\") pod \"91e98444-307d-4f9f-b3a3-9550ef1b2eb7\" (UID: \"91e98444-307d-4f9f-b3a3-9550ef1b2eb7\") "
Sep 16 17:07:50.566710 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:50.566690    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91e98444-307d-4f9f-b3a3-9550ef1b2eb7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91e98444-307d-4f9f-b3a3-9550ef1b2eb7" (UID: "91e98444-307d-4f9f-b3a3-9550ef1b2eb7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:07:50.667545 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:50.667510    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91e98444-307d-4f9f-b3a3-9550ef1b2eb7-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:07:51.288745 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:51.288666    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"
Sep 16 17:07:51.288745 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:51.288685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj" event={"ID":"91e98444-307d-4f9f-b3a3-9550ef1b2eb7","Type":"ContainerDied","Data":"8e755c2a37d04c6ef9805b4c94648bb9ab82e9327709033cd37124d60789e66c"}
Sep 16 17:07:51.288745 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:51.288739    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e755c2a37d04c6ef9805b4c94648bb9ab82e9327709033cd37124d60789e66c"
Sep 16 17:07:51.385635 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:51.385597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:07:52.116743 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:07:52.116703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:03.471832 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.471781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:03.503879 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.503834    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"]
Sep 16 17:08:03.504382 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.504307    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj" podUID="b1875a44-4c8f-4d76-8f19-4c77314f9558" containerName="payload-processing" containerID="cri-o://268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771" gracePeriod=30
Sep 16 17:08:03.515225 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.515198    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"]
Sep 16 17:08:03.515447 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.515421    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn" podUID="299d0d1c-7c7f-41f7-9103-76723f28715b" containerName="payload-pre-processing" containerID="cri-o://b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b" gracePeriod=30
Sep 16 17:08:03.552046 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.552011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:03.631883 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.631836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:03.703068 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.703035    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"]
Sep 16 17:08:03.708626 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.708595    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:03.709361 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.709336    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-9ac315-wnqdj"]
Sep 16 17:08:03.773350 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.773316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:03.792585 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.792563    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:08:03.795857 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.795819    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:08:03.890210 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.890175    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-498zh\" (UniqueName: \"kubernetes.io/projected/b1875a44-4c8f-4d76-8f19-4c77314f9558-kube-api-access-498zh\") pod \"b1875a44-4c8f-4d76-8f19-4c77314f9558\" (UID: \"b1875a44-4c8f-4d76-8f19-4c77314f9558\") "
Sep 16 17:08:03.890400 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.890233    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpqpg\" (UniqueName: \"kubernetes.io/projected/299d0d1c-7c7f-41f7-9103-76723f28715b-kube-api-access-bpqpg\") pod \"299d0d1c-7c7f-41f7-9103-76723f28715b\" (UID: \"299d0d1c-7c7f-41f7-9103-76723f28715b\") "
Sep 16 17:08:03.890476 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.890408    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/299d0d1c-7c7f-41f7-9103-76723f28715b-plugins-config-volume\") pod \"299d0d1c-7c7f-41f7-9103-76723f28715b\" (UID: \"299d0d1c-7c7f-41f7-9103-76723f28715b\") "
Sep 16 17:08:03.890476 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.890457    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1875a44-4c8f-4d76-8f19-4c77314f9558-plugins-config-volume\") pod \"b1875a44-4c8f-4d76-8f19-4c77314f9558\" (UID: \"b1875a44-4c8f-4d76-8f19-4c77314f9558\") "
Sep 16 17:08:03.890852 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.890826    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/299d0d1c-7c7f-41f7-9103-76723f28715b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "299d0d1c-7c7f-41f7-9103-76723f28715b" (UID: "299d0d1c-7c7f-41f7-9103-76723f28715b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:08:03.890961 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.890898    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1875a44-4c8f-4d76-8f19-4c77314f9558-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b1875a44-4c8f-4d76-8f19-4c77314f9558" (UID: "b1875a44-4c8f-4d76-8f19-4c77314f9558"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:08:03.890961 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.890943    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/299d0d1c-7c7f-41f7-9103-76723f28715b-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:08:03.892293 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.892271    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/299d0d1c-7c7f-41f7-9103-76723f28715b-kube-api-access-bpqpg" (OuterVolumeSpecName: "kube-api-access-bpqpg") pod "299d0d1c-7c7f-41f7-9103-76723f28715b" (UID: "299d0d1c-7c7f-41f7-9103-76723f28715b"). InnerVolumeSpecName "kube-api-access-bpqpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:08:03.892415 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.892369    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1875a44-4c8f-4d76-8f19-4c77314f9558-kube-api-access-498zh" (OuterVolumeSpecName: "kube-api-access-498zh") pod "b1875a44-4c8f-4d76-8f19-4c77314f9558" (UID: "b1875a44-4c8f-4d76-8f19-4c77314f9558"). InnerVolumeSpecName "kube-api-access-498zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:08:03.992502 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.992455    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1875a44-4c8f-4d76-8f19-4c77314f9558-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:08:03.992502 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.992493    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-498zh\" (UniqueName: \"kubernetes.io/projected/b1875a44-4c8f-4d76-8f19-4c77314f9558-kube-api-access-498zh\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:08:03.992502 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:03.992509    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpqpg\" (UniqueName: \"kubernetes.io/projected/299d0d1c-7c7f-41f7-9103-76723f28715b-kube-api-access-bpqpg\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:08:04.069056 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.068970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:04.168981 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.168940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:04.374782 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.374701    2577 generic.go:356] "Generic (PLEG): container finished" podID="299d0d1c-7c7f-41f7-9103-76723f28715b" containerID="b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b" exitCode=0
Sep 16 17:08:04.374782 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.374751    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"
Sep 16 17:08:04.374988 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.374784    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn" event={"ID":"299d0d1c-7c7f-41f7-9103-76723f28715b","Type":"ContainerDied","Data":"b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b"}
Sep 16 17:08:04.374988 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.374841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn" event={"ID":"299d0d1c-7c7f-41f7-9103-76723f28715b","Type":"ContainerDied","Data":"25dcca1ea53cc7cef5f2e5435f0e835f1bad507d4d3507657af2da49051f475a"}
Sep 16 17:08:04.374988 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.374867    2577 scope.go:117] "RemoveContainer" containerID="b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b"
Sep 16 17:08:04.376270 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.376246    2577 generic.go:356] "Generic (PLEG): container finished" podID="b1875a44-4c8f-4d76-8f19-4c77314f9558" containerID="268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771" exitCode=0
Sep 16 17:08:04.376390 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.376297    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"
Sep 16 17:08:04.376390 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.376375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj" event={"ID":"b1875a44-4c8f-4d76-8f19-4c77314f9558","Type":"ContainerDied","Data":"268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771"}
Sep 16 17:08:04.376518 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.376466    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj" event={"ID":"b1875a44-4c8f-4d76-8f19-4c77314f9558","Type":"ContainerDied","Data":"88419cbb4750b1aa798bbdf22e19eea92345cd181b9fa2afe775feff7a95298c"}
Sep 16 17:08:04.385176 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.385158    2577 scope.go:117] "RemoveContainer" containerID="b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b"
Sep 16 17:08:04.385485 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:08:04.385454    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b\": container with ID starting with b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b not found: ID does not exist" containerID="b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b"
Sep 16 17:08:04.385615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.385496    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b"} err="failed to get container status \"b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b\": rpc error: code = NotFound desc = could not find container \"b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b\": container with ID starting with b8f7b61e153854a97e591acd8195bd2232bd262bc7b50a66d05ca313d99cc81b not found: ID does not exist"
Sep 16 17:08:04.385615 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.385520    2577 scope.go:117] "RemoveContainer" containerID="268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771"
Sep 16 17:08:04.396054 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.396033    2577 scope.go:117] "RemoveContainer" containerID="268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771"
Sep 16 17:08:04.396288 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:08:04.396271    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771\": container with ID starting with 268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771 not found: ID does not exist" containerID="268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771"
Sep 16 17:08:04.396331 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.396296    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771"} err="failed to get container status \"268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771\": rpc error: code = NotFound desc = could not find container \"268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771\": container with ID starting with 268e33e29dcd944507789d3de23a854cf918edc13ba5220b9fd49d9c56e74771 not found: ID does not exist"
Sep 16 17:08:04.401465 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.401443    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"]
Sep 16 17:08:04.405359 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.405337    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-9ac315-67fbd9498b-rlhwn"]
Sep 16 17:08:04.415583 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.415561    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"]
Sep 16 17:08:04.420508 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.420489    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-9ac315-6fdcf764cc-6jzxj"]
Sep 16 17:08:04.873242 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.873207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:04.977849 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.977817    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="299d0d1c-7c7f-41f7-9103-76723f28715b" path="/var/lib/kubelet/pods/299d0d1c-7c7f-41f7-9103-76723f28715b/volumes"
Sep 16 17:08:04.978198 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.978184    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91e98444-307d-4f9f-b3a3-9550ef1b2eb7" path="/var/lib/kubelet/pods/91e98444-307d-4f9f-b3a3-9550ef1b2eb7/volumes"
Sep 16 17:08:04.978637 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:04.978620    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1875a44-4c8f-4d76-8f19-4c77314f9558" path="/var/lib/kubelet/pods/b1875a44-4c8f-4d76-8f19-4c77314f9558/volumes"
Sep 16 17:08:05.637920 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:05.637886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:06.404120 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:06.404077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:09.147427 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:09.147395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:19.929196 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:19.929153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:40.072304 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:40.072265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:46.203288 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:46.203256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:46.602037 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:46.601954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:08:52.522191 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:08:52.522152    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:06.679122 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:06.679077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:06.780855 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:06.780826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:06.945840 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:06.945742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:07.297499 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.297406    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"]
Sep 16 17:09:07.297977 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.297949    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91e98444-307d-4f9f-b3a3-9550ef1b2eb7" containerName="revoke-keys"
Sep 16 17:09:07.297977 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.297973    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="91e98444-307d-4f9f-b3a3-9550ef1b2eb7" containerName="revoke-keys"
Sep 16 17:09:07.298148 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.297995    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1875a44-4c8f-4d76-8f19-4c77314f9558" containerName="payload-processing"
Sep 16 17:09:07.298148 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.298004    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1875a44-4c8f-4d76-8f19-4c77314f9558" containerName="payload-processing"
Sep 16 17:09:07.298148 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.298022    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="299d0d1c-7c7f-41f7-9103-76723f28715b" containerName="payload-pre-processing"
Sep 16 17:09:07.298148 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.298031    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="299d0d1c-7c7f-41f7-9103-76723f28715b" containerName="payload-pre-processing"
Sep 16 17:09:07.298148 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.298131    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1875a44-4c8f-4d76-8f19-4c77314f9558" containerName="payload-processing"
Sep 16 17:09:07.298148 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.298141    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="299d0d1c-7c7f-41f7-9103-76723f28715b" containerName="payload-pre-processing"
Sep 16 17:09:07.298435 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.298155    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="91e98444-307d-4f9f-b3a3-9550ef1b2eb7" containerName="revoke-keys"
Sep 16 17:09:07.301646 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.301624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"
Sep 16 17:09:07.304030 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.304005    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:09:07.304030 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.304005    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:09:07.309372 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.309310    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"]
Sep 16 17:09:07.364295 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.364268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f94ec59-e520-4feb-bc19-02f201069e40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd\" (UID: \"4f94ec59-e520-4feb-bc19-02f201069e40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"
Sep 16 17:09:07.465568 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.465518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f94ec59-e520-4feb-bc19-02f201069e40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd\" (UID: \"4f94ec59-e520-4feb-bc19-02f201069e40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"
Sep 16 17:09:07.466128 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.466106    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f94ec59-e520-4feb-bc19-02f201069e40-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd\" (UID: \"4f94ec59-e520-4feb-bc19-02f201069e40\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"
Sep 16 17:09:07.581914 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.581822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:07.616434 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.616396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"
Sep 16 17:09:07.888226 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:07.888201    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"]
Sep 16 17:09:07.890134 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:09:07.890104    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f94ec59_e520_4feb_bc19_02f201069e40.slice/crio-e9e38ffc451a32c36dd289f57d6c50f61e40a99b557f5ec54b993b47b0e4c4ca WatchSource:0}: Error finding container e9e38ffc451a32c36dd289f57d6c50f61e40a99b557f5ec54b993b47b0e4c4ca: Status 404 returned error can't find the container with id e9e38ffc451a32c36dd289f57d6c50f61e40a99b557f5ec54b993b47b0e4c4ca
Sep 16 17:09:08.325128 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:08.325036    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:08.756039 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:08.756004    2577 generic.go:356] "Generic (PLEG): container finished" podID="4f94ec59-e520-4feb-bc19-02f201069e40" containerID="f454cdfaf03d18f116dee980b21f145d23795b9580d3b4c4d57b83518376980c" exitCode=0
Sep 16 17:09:08.756219 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:08.756057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd" event={"ID":"4f94ec59-e520-4feb-bc19-02f201069e40","Type":"ContainerDied","Data":"f454cdfaf03d18f116dee980b21f145d23795b9580d3b4c4d57b83518376980c"}
Sep 16 17:09:08.756219 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:08.756108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd" event={"ID":"4f94ec59-e520-4feb-bc19-02f201069e40","Type":"ContainerStarted","Data":"e9e38ffc451a32c36dd289f57d6c50f61e40a99b557f5ec54b993b47b0e4c4ca"}
Sep 16 17:09:09.292940 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:09.292884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:09.892739 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:09.892716    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"
Sep 16 17:09:09.990795 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:09.990764    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f94ec59-e520-4feb-bc19-02f201069e40-maas-api-service-ca\") pod \"4f94ec59-e520-4feb-bc19-02f201069e40\" (UID: \"4f94ec59-e520-4feb-bc19-02f201069e40\") "
Sep 16 17:09:09.991119 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:09.991098    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f94ec59-e520-4feb-bc19-02f201069e40-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f94ec59-e520-4feb-bc19-02f201069e40" (UID: "4f94ec59-e520-4feb-bc19-02f201069e40"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:10.041958 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:10.041840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:10.091909 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:10.091880    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f94ec59-e520-4feb-bc19-02f201069e40-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:10.772247 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:10.772205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd" event={"ID":"4f94ec59-e520-4feb-bc19-02f201069e40","Type":"ContainerDied","Data":"e9e38ffc451a32c36dd289f57d6c50f61e40a99b557f5ec54b993b47b0e4c4ca"}
Sep 16 17:09:10.772247 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:10.772233    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"
Sep 16 17:09:10.772802 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:10.772276    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9e38ffc451a32c36dd289f57d6c50f61e40a99b557f5ec54b993b47b0e4c4ca"
Sep 16 17:09:10.890957 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:10.890923    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:17.907912 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:17.907826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:17.944070 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:17.944041    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"]
Sep 16 17:09:17.944313 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:17.944289    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2" podUID="bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc" containerName="payload-pre-processing" containerID="cri-o://981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e" gracePeriod=30
Sep 16 17:09:18.008485 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.008449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:18.108578 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.108511    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"]
Sep 16 17:09:18.109570 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.109451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:18.111964 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.111939    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-6586c3-n4fhd"]
Sep 16 17:09:18.201378 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.201353    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:09:18.208648 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.208622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:18.258138 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.258108    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg8sc\" (UniqueName: \"kubernetes.io/projected/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-kube-api-access-sg8sc\") pod \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\" (UID: \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\") "
Sep 16 17:09:18.258138 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.258140    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-plugins-config-volume\") pod \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\" (UID: \"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc\") "
Sep 16 17:09:18.258558 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.258516    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc" (UID: "bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:18.260201 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.260181    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-kube-api-access-sg8sc" (OuterVolumeSpecName: "kube-api-access-sg8sc") pod "bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc" (UID: "bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc"). InnerVolumeSpecName "kube-api-access-sg8sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:18.359094 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.359064    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg8sc\" (UniqueName: \"kubernetes.io/projected/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-kube-api-access-sg8sc\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:18.359094 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.359089    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:18.541492 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.541398    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:18.639766 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.639725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:18.826924 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.826844    2577 generic.go:356] "Generic (PLEG): container finished" podID="bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc" containerID="981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e" exitCode=0
Sep 16 17:09:18.826924 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.826889    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"
Sep 16 17:09:18.827138 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.826934    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2" event={"ID":"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc","Type":"ContainerDied","Data":"981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e"}
Sep 16 17:09:18.827138 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.826992    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2" event={"ID":"bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc","Type":"ContainerDied","Data":"c0c14e00ec251c9e8b785bc8be6ce06e2596fad218c1de3d28f8e9fd51724a39"}
Sep 16 17:09:18.827138 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.827030    2577 scope.go:117] "RemoveContainer" containerID="981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e"
Sep 16 17:09:18.837159 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.837139    2577 scope.go:117] "RemoveContainer" containerID="981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e"
Sep 16 17:09:18.837402 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:09:18.837383    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e\": container with ID starting with 981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e not found: ID does not exist" containerID="981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e"
Sep 16 17:09:18.837448 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.837410    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e"} err="failed to get container status \"981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e\": rpc error: code = NotFound desc = could not find container \"981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e\": container with ID starting with 981384172c16affb522e0c665c6baaf98995f2e7ac36171123d2bc678d26373e not found: ID does not exist"
Sep 16 17:09:18.849512 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.849485    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"]
Sep 16 17:09:18.856971 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.856950    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6586c3-8f6474b86-gb2w2"]
Sep 16 17:09:18.977445 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.977414    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f94ec59-e520-4feb-bc19-02f201069e40" path="/var/lib/kubelet/pods/4f94ec59-e520-4feb-bc19-02f201069e40/volumes"
Sep 16 17:09:18.977804 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:18.977793    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc" path="/var/lib/kubelet/pods/bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc/volumes"
Sep 16 17:09:19.354954 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:19.354906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:20.168059 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:20.168022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:24.315291 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.315254    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:24.385404 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.385367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:24.563456 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.563421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:24.884325 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.884285    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"]
Sep 16 17:09:24.884798 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.884772    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc" containerName="payload-pre-processing"
Sep 16 17:09:24.884798 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.884797    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc" containerName="payload-pre-processing"
Sep 16 17:09:24.884962 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.884816    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f94ec59-e520-4feb-bc19-02f201069e40" containerName="revoke-keys"
Sep 16 17:09:24.884962 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.884826    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f94ec59-e520-4feb-bc19-02f201069e40" containerName="revoke-keys"
Sep 16 17:09:24.884962 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.884899    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bca3d81a-ad1b-4da2-b0a9-f237ef30b2fc" containerName="payload-pre-processing"
Sep 16 17:09:24.884962 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.884907    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f94ec59-e520-4feb-bc19-02f201069e40" containerName="revoke-keys"
Sep 16 17:09:24.889125 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.889108    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"
Sep 16 17:09:24.891439 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.891418    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:09:24.891582 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.891490    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:09:24.897226 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:24.897181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"]
Sep 16 17:09:25.010940 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.010881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:25.019162 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.019132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62885e1-395d-4638-a59e-a4a96fa40a77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg\" (UID: \"b62885e1-395d-4638-a59e-a4a96fa40a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"
Sep 16 17:09:25.119910 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.119876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62885e1-395d-4638-a59e-a4a96fa40a77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg\" (UID: \"b62885e1-395d-4638-a59e-a4a96fa40a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"
Sep 16 17:09:25.120514 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.120491    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62885e1-395d-4638-a59e-a4a96fa40a77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg\" (UID: \"b62885e1-395d-4638-a59e-a4a96fa40a77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"
Sep 16 17:09:25.201616 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.201578    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"
Sep 16 17:09:25.472489 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.472466    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"]
Sep 16 17:09:25.475567 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:09:25.475518    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62885e1_395d_4638_a59e_a4a96fa40a77.slice/crio-79a2701dd67eed128437960bf11d47e459bde16a568f5269c8fd9d2169221eb8 WatchSource:0}: Error finding container 79a2701dd67eed128437960bf11d47e459bde16a568f5269c8fd9d2169221eb8: Status 404 returned error can't find the container with id 79a2701dd67eed128437960bf11d47e459bde16a568f5269c8fd9d2169221eb8
Sep 16 17:09:25.870850 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.870756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg" event={"ID":"b62885e1-395d-4638-a59e-a4a96fa40a77","Type":"ContainerStarted","Data":"1c13af07615f9f91de81dc0ded28a92e7f9c39093586b8ce8696a095b4e9088e"}
Sep 16 17:09:25.871013 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.870864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg" event={"ID":"b62885e1-395d-4638-a59e-a4a96fa40a77","Type":"ContainerStarted","Data":"79a2701dd67eed128437960bf11d47e459bde16a568f5269c8fd9d2169221eb8"}
Sep 16 17:09:25.908964 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.908915    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg" podStartSLOduration=1.908900537 podStartE2EDuration="1.908900537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:09:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:09:25.906130345 +0000 UTC m=+1981.403003468" watchObservedRunningTime="2026-09-16 17:09:25.908900537 +0000 UTC m=+1981.405773663"
Sep 16 17:09:25.970442 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:25.970403    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:26.665939 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:26.665893    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:26.877903 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:26.877867    2577 generic.go:356] "Generic (PLEG): container finished" podID="b62885e1-395d-4638-a59e-a4a96fa40a77" containerID="1c13af07615f9f91de81dc0ded28a92e7f9c39093586b8ce8696a095b4e9088e" exitCode=0
Sep 16 17:09:26.878067 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:26.877929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg" event={"ID":"b62885e1-395d-4638-a59e-a4a96fa40a77","Type":"ContainerDied","Data":"1c13af07615f9f91de81dc0ded28a92e7f9c39093586b8ce8696a095b4e9088e"}
Sep 16 17:09:27.394109 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:27.394080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:28.018394 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:28.018370    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"
Sep 16 17:09:28.149034 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:28.149009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62885e1-395d-4638-a59e-a4a96fa40a77-maas-api-service-ca\") pod \"b62885e1-395d-4638-a59e-a4a96fa40a77\" (UID: \"b62885e1-395d-4638-a59e-a4a96fa40a77\") "
Sep 16 17:09:28.149363 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:28.149339    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b62885e1-395d-4638-a59e-a4a96fa40a77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b62885e1-395d-4638-a59e-a4a96fa40a77" (UID: "b62885e1-395d-4638-a59e-a4a96fa40a77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:28.250064 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:28.250033    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b62885e1-395d-4638-a59e-a4a96fa40a77-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:28.892180 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:28.892150    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"
Sep 16 17:09:28.892180 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:28.892171    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg" event={"ID":"b62885e1-395d-4638-a59e-a4a96fa40a77","Type":"ContainerDied","Data":"79a2701dd67eed128437960bf11d47e459bde16a568f5269c8fd9d2169221eb8"}
Sep 16 17:09:28.892363 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:28.892218    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79a2701dd67eed128437960bf11d47e459bde16a568f5269c8fd9d2169221eb8"
Sep 16 17:09:35.362721 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.362671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:35.388624 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.388593    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"]
Sep 16 17:09:35.388831 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.388810    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd" podUID="5896cf57-8f2f-4f1f-9d51-bd2fdc00231a" containerName="payload-processing" containerID="cri-o://0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be" gracePeriod=30
Sep 16 17:09:35.396209 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.396180    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"]
Sep 16 17:09:35.396473 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.396445    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b" podUID="bc7e98c8-31fe-45be-8383-716bc44e5f9f" containerName="payload-pre-processing" containerID="cri-o://0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6" gracePeriod=30
Sep 16 17:09:35.462242 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.462197    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:35.563933 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.563903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:35.648994 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.648884    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"]
Sep 16 17:09:35.651840 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.651400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:35.653037 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.652868    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-52fd8c-2nmpg"]
Sep 16 17:09:35.681860 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.681837    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 17:09:35.685321 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.685300    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 17:09:35.711593 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.711548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:35.823384 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.823348    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klswx\" (UniqueName: \"kubernetes.io/projected/bc7e98c8-31fe-45be-8383-716bc44e5f9f-kube-api-access-klswx\") pod \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\" (UID: \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\") "
Sep 16 17:09:35.823588 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.823408    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc7e98c8-31fe-45be-8383-716bc44e5f9f-plugins-config-volume\") pod \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\" (UID: \"bc7e98c8-31fe-45be-8383-716bc44e5f9f\") "
Sep 16 17:09:35.823588 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.823473    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-plugins-config-volume\") pod \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\" (UID: \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\") "
Sep 16 17:09:35.823588 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.823516    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg79w\" (UniqueName: \"kubernetes.io/projected/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-kube-api-access-xg79w\") pod \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\" (UID: \"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a\") "
Sep 16 17:09:35.823916 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.823883    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc7e98c8-31fe-45be-8383-716bc44e5f9f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc7e98c8-31fe-45be-8383-716bc44e5f9f" (UID: "bc7e98c8-31fe-45be-8383-716bc44e5f9f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:35.824037 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.823915    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5896cf57-8f2f-4f1f-9d51-bd2fdc00231a" (UID: "5896cf57-8f2f-4f1f-9d51-bd2fdc00231a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:35.825572 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.825551    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc7e98c8-31fe-45be-8383-716bc44e5f9f-kube-api-access-klswx" (OuterVolumeSpecName: "kube-api-access-klswx") pod "bc7e98c8-31fe-45be-8383-716bc44e5f9f" (UID: "bc7e98c8-31fe-45be-8383-716bc44e5f9f"). InnerVolumeSpecName "kube-api-access-klswx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:35.825697 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.825681    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-kube-api-access-xg79w" (OuterVolumeSpecName: "kube-api-access-xg79w") pod "5896cf57-8f2f-4f1f-9d51-bd2fdc00231a" (UID: "5896cf57-8f2f-4f1f-9d51-bd2fdc00231a"). InnerVolumeSpecName "kube-api-access-xg79w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:35.925019 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.924935    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klswx\" (UniqueName: \"kubernetes.io/projected/bc7e98c8-31fe-45be-8383-716bc44e5f9f-kube-api-access-klswx\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:35.925019 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.924964    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc7e98c8-31fe-45be-8383-716bc44e5f9f-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:35.925019 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.924975    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:35.925019 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.924984    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg79w\" (UniqueName: \"kubernetes.io/projected/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a-kube-api-access-xg79w\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:35.935186 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.935153    2577 generic.go:356] "Generic (PLEG): container finished" podID="bc7e98c8-31fe-45be-8383-716bc44e5f9f" containerID="0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6" exitCode=0
Sep 16 17:09:35.935353 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.935209    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"
Sep 16 17:09:35.935353 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.935249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b" event={"ID":"bc7e98c8-31fe-45be-8383-716bc44e5f9f","Type":"ContainerDied","Data":"0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6"}
Sep 16 17:09:35.935353 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.935301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b" event={"ID":"bc7e98c8-31fe-45be-8383-716bc44e5f9f","Type":"ContainerDied","Data":"c4747818b220c331aae2c8ed47f4de81215278f1fb86983463076ef6be37e3f6"}
Sep 16 17:09:35.935353 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.935332    2577 scope.go:117] "RemoveContainer" containerID="0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6"
Sep 16 17:09:35.936844 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.936820    2577 generic.go:356] "Generic (PLEG): container finished" podID="5896cf57-8f2f-4f1f-9d51-bd2fdc00231a" containerID="0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be" exitCode=0
Sep 16 17:09:35.936982 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.936860    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"
Sep 16 17:09:35.936982 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.936857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd" event={"ID":"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a","Type":"ContainerDied","Data":"0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be"}
Sep 16 17:09:35.936982 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.936912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd" event={"ID":"5896cf57-8f2f-4f1f-9d51-bd2fdc00231a","Type":"ContainerDied","Data":"c42d6aa92f39855498a899bf14b61f4fd75d62781137e5a112a2b68c953c6532"}
Sep 16 17:09:35.946601 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.946582    2577 scope.go:117] "RemoveContainer" containerID="0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6"
Sep 16 17:09:35.946929 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:09:35.946901    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6\": container with ID starting with 0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6 not found: ID does not exist" containerID="0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6"
Sep 16 17:09:35.947011 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.946937    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6"} err="failed to get container status \"0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6\": rpc error: code = NotFound desc = could not find container \"0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6\": container with ID starting with 0686320efb4f3416b0fafd91e0c8e23d4b280e36cac6ede51ce694e544103ba6 not found: ID does not exist"
Sep 16 17:09:35.947011 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.946955    2577 scope.go:117] "RemoveContainer" containerID="0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be"
Sep 16 17:09:35.955588 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.955569    2577 scope.go:117] "RemoveContainer" containerID="0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be"
Sep 16 17:09:35.955868 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:09:35.955850    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be\": container with ID starting with 0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be not found: ID does not exist" containerID="0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be"
Sep 16 17:09:35.955930 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.955878    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be"} err="failed to get container status \"0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be\": rpc error: code = NotFound desc = could not find container \"0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be\": container with ID starting with 0648d6cfb1962561c1df4255cab4713a31fdc0b7e3cbed1b4fd0e11ab8b125be not found: ID does not exist"
Sep 16 17:09:35.977576 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.977551    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"]
Sep 16 17:09:35.984717 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:35.984694    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-52fd8c-56d789b695-xc8hd"]
Sep 16 17:09:36.010374 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:36.010338    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"]
Sep 16 17:09:36.021032 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:36.021007    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-52fd8c-5889c878f8-bp69b"]
Sep 16 17:09:36.048726 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:36.048691    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:36.157581 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:36.157511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:36.791512 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:36.791481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:36.983868 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:36.983832    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5896cf57-8f2f-4f1f-9d51-bd2fdc00231a" path="/var/lib/kubelet/pods/5896cf57-8f2f-4f1f-9d51-bd2fdc00231a/volumes"
Sep 16 17:09:36.984185 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:36.984173    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b62885e1-395d-4638-a59e-a4a96fa40a77" path="/var/lib/kubelet/pods/b62885e1-395d-4638-a59e-a4a96fa40a77/volumes"
Sep 16 17:09:36.984487 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:36.984475    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc7e98c8-31fe-45be-8383-716bc44e5f9f" path="/var/lib/kubelet/pods/bc7e98c8-31fe-45be-8383-716bc44e5f9f/volumes"
Sep 16 17:09:42.387121 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:42.387080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:42.531567 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:42.531507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:42.775431 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:42.775340    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:43.054868 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.054793    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"]
Sep 16 17:09:43.055189 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055175    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc7e98c8-31fe-45be-8383-716bc44e5f9f" containerName="payload-pre-processing"
Sep 16 17:09:43.055233 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055191    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc7e98c8-31fe-45be-8383-716bc44e5f9f" containerName="payload-pre-processing"
Sep 16 17:09:43.055233 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055204    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b62885e1-395d-4638-a59e-a4a96fa40a77" containerName="revoke-keys"
Sep 16 17:09:43.055233 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055210    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b62885e1-395d-4638-a59e-a4a96fa40a77" containerName="revoke-keys"
Sep 16 17:09:43.055233 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055225    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5896cf57-8f2f-4f1f-9d51-bd2fdc00231a" containerName="payload-processing"
Sep 16 17:09:43.055233 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055231    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5896cf57-8f2f-4f1f-9d51-bd2fdc00231a" containerName="payload-processing"
Sep 16 17:09:43.055389 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055287    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5896cf57-8f2f-4f1f-9d51-bd2fdc00231a" containerName="payload-processing"
Sep 16 17:09:43.055389 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055296    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc7e98c8-31fe-45be-8383-716bc44e5f9f" containerName="payload-pre-processing"
Sep 16 17:09:43.055389 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.055305    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b62885e1-395d-4638-a59e-a4a96fa40a77" containerName="revoke-keys"
Sep 16 17:09:43.059707 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.059691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"
Sep 16 17:09:43.062308 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.062289    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 17:09:43.062427 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.062368    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-69rc9\""
Sep 16 17:09:43.066846 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.066827    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"]
Sep 16 17:09:43.093176 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.093149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e805b1f-c347-41c0-bb96-3e06e371a95c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72\" (UID: \"8e805b1f-c347-41c0-bb96-3e06e371a95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"
Sep 16 17:09:43.194371 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.194321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e805b1f-c347-41c0-bb96-3e06e371a95c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72\" (UID: \"8e805b1f-c347-41c0-bb96-3e06e371a95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"
Sep 16 17:09:43.194967 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.194944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e805b1f-c347-41c0-bb96-3e06e371a95c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72\" (UID: \"8e805b1f-c347-41c0-bb96-3e06e371a95c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"
Sep 16 17:09:43.371601 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.371496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"
Sep 16 17:09:43.645432 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.645406    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"]
Sep 16 17:09:43.647233 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:09:43.647202    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e805b1f_c347_41c0_bb96_3e06e371a95c.slice/crio-acb8867131c1a29e3a48f8589aeeb8105d8bdb39c69f723f324d94b5001408ed WatchSource:0}: Error finding container acb8867131c1a29e3a48f8589aeeb8105d8bdb39c69f723f324d94b5001408ed: Status 404 returned error can't find the container with id acb8867131c1a29e3a48f8589aeeb8105d8bdb39c69f723f324d94b5001408ed
Sep 16 17:09:43.973887 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.973855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:43.988264 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.988227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72" event={"ID":"8e805b1f-c347-41c0-bb96-3e06e371a95c","Type":"ContainerStarted","Data":"f4e9a3958592af37143a0299e6d95b5ab608ba30855bb50ba2fabaa82b9ce5bd"}
Sep 16 17:09:43.988384 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:43.988286    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72" event={"ID":"8e805b1f-c347-41c0-bb96-3e06e371a95c","Type":"ContainerStarted","Data":"acb8867131c1a29e3a48f8589aeeb8105d8bdb39c69f723f324d94b5001408ed"}
Sep 16 17:09:44.027030 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:44.021610    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72" podStartSLOduration=1.021590452 podStartE2EDuration="1.021590452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:09:44.018776419 +0000 UTC m=+1999.515649542" watchObservedRunningTime="2026-09-16 17:09:44.021590452 +0000 UTC m=+1999.518463577"
Sep 16 17:09:44.710916 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:44.710878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:44.994741 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:44.994660    2577 generic.go:356] "Generic (PLEG): container finished" podID="8e805b1f-c347-41c0-bb96-3e06e371a95c" containerID="f4e9a3958592af37143a0299e6d95b5ab608ba30855bb50ba2fabaa82b9ce5bd" exitCode=0
Sep 16 17:09:44.994883 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:44.994747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72" event={"ID":"8e805b1f-c347-41c0-bb96-3e06e371a95c","Type":"ContainerDied","Data":"f4e9a3958592af37143a0299e6d95b5ab608ba30855bb50ba2fabaa82b9ce5bd"}
Sep 16 17:09:46.129160 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:46.129136    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"
Sep 16 17:09:46.217156 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:46.217125    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e805b1f-c347-41c0-bb96-3e06e371a95c-maas-api-service-ca\") pod \"8e805b1f-c347-41c0-bb96-3e06e371a95c\" (UID: \"8e805b1f-c347-41c0-bb96-3e06e371a95c\") "
Sep 16 17:09:46.217419 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:46.217392    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e805b1f-c347-41c0-bb96-3e06e371a95c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e805b1f-c347-41c0-bb96-3e06e371a95c" (UID: "8e805b1f-c347-41c0-bb96-3e06e371a95c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:46.317973 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:46.317897    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e805b1f-c347-41c0-bb96-3e06e371a95c-maas-api-service-ca\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:47.007945 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:47.007918    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"
Sep 16 17:09:47.008085 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:47.007914    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72" event={"ID":"8e805b1f-c347-41c0-bb96-3e06e371a95c","Type":"ContainerDied","Data":"acb8867131c1a29e3a48f8589aeeb8105d8bdb39c69f723f324d94b5001408ed"}
Sep 16 17:09:47.008085 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:47.008031    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acb8867131c1a29e3a48f8589aeeb8105d8bdb39c69f723f324d94b5001408ed"
Sep 16 17:09:53.557661 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.557594    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"]
Sep 16 17:09:53.564462 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.564428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:53.564802 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.564780    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-34d301-29826300-5pwhk"]
Sep 16 17:09:53.597340 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.597311    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"]
Sep 16 17:09:53.597605 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.597578    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k" podUID="5d1c623a-888e-404e-9192-520d547daa22" containerName="payload-pre-processing" containerID="cri-o://a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790" gracePeriod=30
Sep 16 17:09:53.662349 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.662306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:53.763726 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.763394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:53.764364 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.764341    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"]
Sep 16 17:09:53.767924 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.767680    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-34d301-rwj72"]
Sep 16 17:09:53.847801 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.847772    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 17:09:53.855638 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.855610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:53.887966 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.887939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d1c623a-888e-404e-9192-520d547daa22-plugins-config-volume\") pod \"5d1c623a-888e-404e-9192-520d547daa22\" (UID: \"5d1c623a-888e-404e-9192-520d547daa22\") "
Sep 16 17:09:53.888108 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.887992    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n8qc\" (UniqueName: \"kubernetes.io/projected/5d1c623a-888e-404e-9192-520d547daa22-kube-api-access-2n8qc\") pod \"5d1c623a-888e-404e-9192-520d547daa22\" (UID: \"5d1c623a-888e-404e-9192-520d547daa22\") "
Sep 16 17:09:53.888402 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.888375    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d1c623a-888e-404e-9192-520d547daa22-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5d1c623a-888e-404e-9192-520d547daa22" (UID: "5d1c623a-888e-404e-9192-520d547daa22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:09:53.890106 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.890078    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1c623a-888e-404e-9192-520d547daa22-kube-api-access-2n8qc" (OuterVolumeSpecName: "kube-api-access-2n8qc") pod "5d1c623a-888e-404e-9192-520d547daa22" (UID: "5d1c623a-888e-404e-9192-520d547daa22"). InnerVolumeSpecName "kube-api-access-2n8qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:09:53.988859 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.988830    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n8qc\" (UniqueName: \"kubernetes.io/projected/5d1c623a-888e-404e-9192-520d547daa22-kube-api-access-2n8qc\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:53.988859 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:53.988856    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d1c623a-888e-404e-9192-520d547daa22-plugins-config-volume\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:09:54.049766 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.049731    2577 generic.go:356] "Generic (PLEG): container finished" podID="5d1c623a-888e-404e-9192-520d547daa22" containerID="a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790" exitCode=0
Sep 16 17:09:54.049952 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.049782    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"
Sep 16 17:09:54.049952 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.049820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k" event={"ID":"5d1c623a-888e-404e-9192-520d547daa22","Type":"ContainerDied","Data":"a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790"}
Sep 16 17:09:54.049952 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.049879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k" event={"ID":"5d1c623a-888e-404e-9192-520d547daa22","Type":"ContainerDied","Data":"e4ddc25d379b07f55168cf0ec79f41125ededed3a17692a1ae71b0e01b909479"}
Sep 16 17:09:54.049952 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.049907    2577 scope.go:117] "RemoveContainer" containerID="a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790"
Sep 16 17:09:54.059608 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.059589    2577 scope.go:117] "RemoveContainer" containerID="a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790"
Sep 16 17:09:54.059881 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:09:54.059863    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790\": container with ID starting with a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790 not found: ID does not exist" containerID="a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790"
Sep 16 17:09:54.059932 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.059889    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790"} err="failed to get container status \"a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790\": rpc error: code = NotFound desc = could not find container \"a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790\": container with ID starting with a6f6d11c6ac4c25e31b01cdefcfb6b24991bd73661ac7df125df4903dc72d790 not found: ID does not exist"
Sep 16 17:09:54.074276 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.074251    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"]
Sep 16 17:09:54.076610 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.076588    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-34d301-54fc64659d-zw78k"]
Sep 16 17:09:54.115652 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.115578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:54.216892 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.216855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:09:54.978355 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.978323    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d1c623a-888e-404e-9192-520d547daa22" path="/var/lib/kubelet/pods/5d1c623a-888e-404e-9192-520d547daa22/volumes"
Sep 16 17:09:54.978738 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.978634    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81d16c34-9219-448d-95a5-1732d7d403a0" path="/var/lib/kubelet/pods/81d16c34-9219-448d-95a5-1732d7d403a0/volumes"
Sep 16 17:09:54.978889 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:09:54.978876    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e805b1f-c347-41c0-bb96-3e06e371a95c" path="/var/lib/kubelet/pods/8e805b1f-c347-41c0-bb96-3e06e371a95c/volumes"
Sep 16 17:10:00.390594 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:00.390557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:10:00.468810 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:00.468772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:10:00.690209 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:00.690123    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:10:01.984072 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:01.984021    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:10:03.361595 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.361544    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-r6jxd"]
Sep 16 17:10:03.361981 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.361758    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd" podUID="007df39e-e454-47cb-8658-364e1ef34b60" containerName="manager" containerID="cri-o://b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79" gracePeriod=30
Sep 16 17:10:03.604775 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.604748    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 17:10:03.674355 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.674323    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glmx6\" (UniqueName: \"kubernetes.io/projected/007df39e-e454-47cb-8658-364e1ef34b60-kube-api-access-glmx6\") pod \"007df39e-e454-47cb-8658-364e1ef34b60\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") "
Sep 16 17:10:03.674539 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.674410    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-metrics-certs\") pod \"007df39e-e454-47cb-8658-364e1ef34b60\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") "
Sep 16 17:10:03.674539 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.674447    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-webhook-cert\") pod \"007df39e-e454-47cb-8658-364e1ef34b60\" (UID: \"007df39e-e454-47cb-8658-364e1ef34b60\") "
Sep 16 17:10:03.676574 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.676505    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/007df39e-e454-47cb-8658-364e1ef34b60-kube-api-access-glmx6" (OuterVolumeSpecName: "kube-api-access-glmx6") pod "007df39e-e454-47cb-8658-364e1ef34b60" (UID: "007df39e-e454-47cb-8658-364e1ef34b60"). InnerVolumeSpecName "kube-api-access-glmx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:10:03.676986 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.676959    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "007df39e-e454-47cb-8658-364e1ef34b60" (UID: "007df39e-e454-47cb-8658-364e1ef34b60"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:10:03.677220 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.677199    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "007df39e-e454-47cb-8658-364e1ef34b60" (UID: "007df39e-e454-47cb-8658-364e1ef34b60"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:10:03.775481 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.775445    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-metrics-certs\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:10:03.775481 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.775476    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/007df39e-e454-47cb-8658-364e1ef34b60-webhook-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:10:03.775698 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:03.775490    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glmx6\" (UniqueName: \"kubernetes.io/projected/007df39e-e454-47cb-8658-364e1ef34b60-kube-api-access-glmx6\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:10:04.108948 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.108852    2577 generic.go:356] "Generic (PLEG): container finished" podID="007df39e-e454-47cb-8658-364e1ef34b60" containerID="b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79" exitCode=0
Sep 16 17:10:04.108948 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.108909    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd"
Sep 16 17:10:04.109156 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.108912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd" event={"ID":"007df39e-e454-47cb-8658-364e1ef34b60","Type":"ContainerDied","Data":"b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79"}
Sep 16 17:10:04.109156 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.109042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-r6jxd" event={"ID":"007df39e-e454-47cb-8658-364e1ef34b60","Type":"ContainerDied","Data":"d7ba777d3624d3ed0e4efb30d4ccc06f5b6abef8b669d1ae243a87a6c6ecd267"}
Sep 16 17:10:04.109156 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.109071    2577 scope.go:117] "RemoveContainer" containerID="b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79"
Sep 16 17:10:04.119118 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.119099    2577 scope.go:117] "RemoveContainer" containerID="b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79"
Sep 16 17:10:04.119376 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:10:04.119357    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79\": container with ID starting with b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79 not found: ID does not exist" containerID="b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79"
Sep 16 17:10:04.119421 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.119386    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79"} err="failed to get container status \"b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79\": rpc error: code = NotFound desc = could not find container \"b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79\": container with ID starting with b67d22364a5caebc370d40e279e6e1057d2f4e0b20bfcffc9807450e0430ba79 not found: ID does not exist"
Sep 16 17:10:04.130725 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.130702    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-r6jxd"]
Sep 16 17:10:04.139957 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.136851    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-r6jxd"]
Sep 16 17:10:04.978788 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:04.978757    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="007df39e-e454-47cb-8658-364e1ef34b60" path="/var/lib/kubelet/pods/007df39e-e454-47cb-8658-364e1ef34b60/volumes"
Sep 16 17:10:25.508077 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:25.508044    2577 scope.go:117] "RemoveContainer" containerID="3470ead5537f8fb893e5f1ff5acb2deed327f0cb55c018977d863b1d828ab4c0"
Sep 16 17:10:25.516468 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:25.516450    2577 scope.go:117] "RemoveContainer" containerID="036aa29503d314e0a07e645fb003be1ec6d6527d56d07b1630f88f78a9034798"
Sep 16 17:10:25.524127 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:25.524109    2577 scope.go:117] "RemoveContainer" containerID="f9aa889706913e1c1f4e7858e0b5f0b33d7a58858418a51996355cb218692b76"
Sep 16 17:10:50.743832 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:50.743740    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:10:50.809039 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:10:50.809007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:11:04.903244 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903208    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-gjbbz"]
Sep 16 17:11:04.903673 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903601    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e805b1f-c347-41c0-bb96-3e06e371a95c" containerName="revoke-keys"
Sep 16 17:11:04.903673 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903614    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e805b1f-c347-41c0-bb96-3e06e371a95c" containerName="revoke-keys"
Sep 16 17:11:04.903673 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903624    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="007df39e-e454-47cb-8658-364e1ef34b60" containerName="manager"
Sep 16 17:11:04.903673 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903630    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="007df39e-e454-47cb-8658-364e1ef34b60" containerName="manager"
Sep 16 17:11:04.903673 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903637    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1c623a-888e-404e-9192-520d547daa22" containerName="payload-pre-processing"
Sep 16 17:11:04.903673 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903642    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1c623a-888e-404e-9192-520d547daa22" containerName="payload-pre-processing"
Sep 16 17:11:04.903886 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903702    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="007df39e-e454-47cb-8658-364e1ef34b60" containerName="manager"
Sep 16 17:11:04.903886 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903712    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1c623a-888e-404e-9192-520d547daa22" containerName="payload-pre-processing"
Sep 16 17:11:04.903886 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.903720    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e805b1f-c347-41c0-bb96-3e06e371a95c" containerName="revoke-keys"
Sep 16 17:11:04.906785 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.906770    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:04.910295 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.910259    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 17:11:04.910435 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.910294    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8kj7n\""
Sep 16 17:11:04.910435 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.910335    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 17:11:04.910435 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.910294    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 17:11:04.917033 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:04.917007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-gjbbz"]
Sep 16 17:11:05.006161 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.006127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-webhook-cert\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.006161 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.006160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-metrics-certs\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.006376 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.006259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqg5p\" (UniqueName: \"kubernetes.io/projected/38915048-e3e3-4482-9407-cb9ab83a0277-kube-api-access-nqg5p\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.107489 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.107448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-webhook-cert\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.107489 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.107488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-metrics-certs\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.107762 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.107617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqg5p\" (UniqueName: \"kubernetes.io/projected/38915048-e3e3-4482-9407-cb9ab83a0277-kube-api-access-nqg5p\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.109969 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.109939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-webhook-cert\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.110101 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.109981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-metrics-certs\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.115938 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.115916    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqg5p\" (UniqueName: \"kubernetes.io/projected/38915048-e3e3-4482-9407-cb9ab83a0277-kube-api-access-nqg5p\") pod \"maas-controller-6bb99c5f94-gjbbz\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") " pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.217196 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.217126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:05.489655 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.489630    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-gjbbz"]
Sep 16 17:11:05.491684 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:11:05.491645    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38915048_e3e3_4482_9407_cb9ab83a0277.slice/crio-5fb3022b73d788c3e5255bc17f25baf3108dcacafba7883cf5a6fd5e6c0e82f8 WatchSource:0}: Error finding container 5fb3022b73d788c3e5255bc17f25baf3108dcacafba7883cf5a6fd5e6c0e82f8: Status 404 returned error can't find the container with id 5fb3022b73d788c3e5255bc17f25baf3108dcacafba7883cf5a6fd5e6c0e82f8
Sep 16 17:11:05.492899 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:05.492878    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:11:06.457472 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:06.457438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz" event={"ID":"38915048-e3e3-4482-9407-cb9ab83a0277","Type":"ContainerStarted","Data":"e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46"}
Sep 16 17:11:06.458024 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:06.457488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz" event={"ID":"38915048-e3e3-4482-9407-cb9ab83a0277","Type":"ContainerStarted","Data":"5fb3022b73d788c3e5255bc17f25baf3108dcacafba7883cf5a6fd5e6c0e82f8"}
Sep 16 17:11:06.458024 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:06.457540    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:06.475572 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:06.475508    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz" podStartSLOduration=1.992246698 podStartE2EDuration="2.475495016s" totalImagesPullingTime="483.248318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:11:06.473877588 +0000 UTC m=+2081.970750729" watchObservedRunningTime="2026-09-16 17:11:06.475495016 +0000 UTC m=+2081.972368139"
Sep 16 17:11:10.720991 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:10.720953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:11:10.780178 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:10.780141    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:11:17.465647 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:17.465616    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:25.086773 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:25.086676    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:11:25.094715 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:25.094694    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:11:25.569892 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:25.569862    2577 scope.go:117] "RemoveContainer" containerID="3f1c48b13c2c7de021b083aa764df7d7bb87af0ea6eba698e784ccd1314c5ac2"
Sep 16 17:11:25.578244 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:25.578225    2577 scope.go:117] "RemoveContainer" containerID="776094a1b4cbfcce9a69a3185e60fcc39fd56e5a75a447633cddd7d9b2be6fc0"
Sep 16 17:11:28.766773 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:28.766728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:11:28.865048 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:28.865015    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-gjbbz"]
Sep 16 17:11:28.865249 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:28.865223    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz" podUID="38915048-e3e3-4482-9407-cb9ab83a0277" containerName="manager" containerID="cri-o://e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46" gracePeriod=30
Sep 16 17:11:29.106309 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.106285    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:29.122105 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.122085    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-webhook-cert\") pod \"38915048-e3e3-4482-9407-cb9ab83a0277\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") "
Sep 16 17:11:29.122193 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.122168    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-metrics-certs\") pod \"38915048-e3e3-4482-9407-cb9ab83a0277\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") "
Sep 16 17:11:29.122193 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.122188    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqg5p\" (UniqueName: \"kubernetes.io/projected/38915048-e3e3-4482-9407-cb9ab83a0277-kube-api-access-nqg5p\") pod \"38915048-e3e3-4482-9407-cb9ab83a0277\" (UID: \"38915048-e3e3-4482-9407-cb9ab83a0277\") "
Sep 16 17:11:29.124592 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.124455    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "38915048-e3e3-4482-9407-cb9ab83a0277" (UID: "38915048-e3e3-4482-9407-cb9ab83a0277"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:11:29.124692 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.124644    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38915048-e3e3-4482-9407-cb9ab83a0277-kube-api-access-nqg5p" (OuterVolumeSpecName: "kube-api-access-nqg5p") pod "38915048-e3e3-4482-9407-cb9ab83a0277" (UID: "38915048-e3e3-4482-9407-cb9ab83a0277"). InnerVolumeSpecName "kube-api-access-nqg5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:11:29.124889 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.124861    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "38915048-e3e3-4482-9407-cb9ab83a0277" (UID: "38915048-e3e3-4482-9407-cb9ab83a0277"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:11:29.223698 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.223660    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-webhook-cert\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:11:29.223698 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.223691    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38915048-e3e3-4482-9407-cb9ab83a0277-metrics-certs\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:11:29.223698 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.223703    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqg5p\" (UniqueName: \"kubernetes.io/projected/38915048-e3e3-4482-9407-cb9ab83a0277-kube-api-access-nqg5p\") on node \"ip-10-0-139-51.ec2.internal\" DevicePath \"\""
Sep 16 17:11:29.590301 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.590266    2577 generic.go:356] "Generic (PLEG): container finished" podID="38915048-e3e3-4482-9407-cb9ab83a0277" containerID="e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46" exitCode=0
Sep 16 17:11:29.590470 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.590315    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz"
Sep 16 17:11:29.590470 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.590354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz" event={"ID":"38915048-e3e3-4482-9407-cb9ab83a0277","Type":"ContainerDied","Data":"e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46"}
Sep 16 17:11:29.590470 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.590414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-gjbbz" event={"ID":"38915048-e3e3-4482-9407-cb9ab83a0277","Type":"ContainerDied","Data":"5fb3022b73d788c3e5255bc17f25baf3108dcacafba7883cf5a6fd5e6c0e82f8"}
Sep 16 17:11:29.590470 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.590440    2577 scope.go:117] "RemoveContainer" containerID="e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46"
Sep 16 17:11:29.603112 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.603066    2577 scope.go:117] "RemoveContainer" containerID="e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46"
Sep 16 17:11:29.603599 ip-10-0-139-51 kubenswrapper[2577]: E0916 17:11:29.603573    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46\": container with ID starting with e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46 not found: ID does not exist" containerID="e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46"
Sep 16 17:11:29.603705 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.603606    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46"} err="failed to get container status \"e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46\": rpc error: code = NotFound desc = could not find container \"e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46\": container with ID starting with e848f2c70f6c38a6adef529263cc3505147b9b27937508d536566b0dc74c9b46 not found: ID does not exist"
Sep 16 17:11:29.614029 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.614007    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-gjbbz"]
Sep 16 17:11:29.617665 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:29.617643    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-gjbbz"]
Sep 16 17:11:30.243786 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.243748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:11:30.348512 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.348476    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-rb9kd"]
Sep 16 17:11:30.348877 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.348858    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38915048-e3e3-4482-9407-cb9ab83a0277" containerName="manager"
Sep 16 17:11:30.348877 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.348877    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="38915048-e3e3-4482-9407-cb9ab83a0277" containerName="manager"
Sep 16 17:11:30.349014 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.348970    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="38915048-e3e3-4482-9407-cb9ab83a0277" containerName="manager"
Sep 16 17:11:30.353282 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.353261    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.357012 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.356987    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8kj7n\""
Sep 16 17:11:30.357012 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.356988    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 17:11:30.357185 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.357047    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 17:11:30.357185 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.357099    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 17:11:30.363337 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.363316    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-rb9kd"]
Sep 16 17:11:30.435419 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.435384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/109a6a82-13a4-46cd-a160-a85e5f8e17ca-webhook-cert\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.435419 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.435420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/109a6a82-13a4-46cd-a160-a85e5f8e17ca-metrics-certs\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.435720 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.435465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gr8v\" (UniqueName: \"kubernetes.io/projected/109a6a82-13a4-46cd-a160-a85e5f8e17ca-kube-api-access-9gr8v\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.536431 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.536357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/109a6a82-13a4-46cd-a160-a85e5f8e17ca-webhook-cert\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.536431 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.536393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/109a6a82-13a4-46cd-a160-a85e5f8e17ca-metrics-certs\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.536431 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.536422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gr8v\" (UniqueName: \"kubernetes.io/projected/109a6a82-13a4-46cd-a160-a85e5f8e17ca-kube-api-access-9gr8v\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.538871 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.538844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/109a6a82-13a4-46cd-a160-a85e5f8e17ca-webhook-cert\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.538978 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.538903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/109a6a82-13a4-46cd-a160-a85e5f8e17ca-metrics-certs\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.544564 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.544542    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gr8v\" (UniqueName: \"kubernetes.io/projected/109a6a82-13a4-46cd-a160-a85e5f8e17ca-kube-api-access-9gr8v\") pod \"maas-controller-6bb99c5f94-rb9kd\" (UID: \"109a6a82-13a4-46cd-a160-a85e5f8e17ca\") " pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.665677 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.665650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:30.939050 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.939022    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bb99c5f94-rb9kd"]
Sep 16 17:11:30.941203 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:11:30.941176    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109a6a82_13a4_46cd_a160_a85e5f8e17ca.slice/crio-b88dd0fa98f0f8f1bf656a83f7de77be2eb29fc28b89db20041807ea264083c3 WatchSource:0}: Error finding container b88dd0fa98f0f8f1bf656a83f7de77be2eb29fc28b89db20041807ea264083c3: Status 404 returned error can't find the container with id b88dd0fa98f0f8f1bf656a83f7de77be2eb29fc28b89db20041807ea264083c3
Sep 16 17:11:30.977029 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:30.976999    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38915048-e3e3-4482-9407-cb9ab83a0277" path="/var/lib/kubelet/pods/38915048-e3e3-4482-9407-cb9ab83a0277/volumes"
Sep 16 17:11:31.605498 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:31.605403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-rb9kd" event={"ID":"109a6a82-13a4-46cd-a160-a85e5f8e17ca","Type":"ContainerStarted","Data":"cc159c3660207bcbd2a0c9324c62f61b25c72a0494b3c7514bc00047cc5e00d5"}
Sep 16 17:11:31.605498 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:31.605491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bb99c5f94-rb9kd" event={"ID":"109a6a82-13a4-46cd-a160-a85e5f8e17ca","Type":"ContainerStarted","Data":"b88dd0fa98f0f8f1bf656a83f7de77be2eb29fc28b89db20041807ea264083c3"}
Sep 16 17:11:31.605891 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:31.605623    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:11:31.625260 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:31.625211    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bb99c5f94-rb9kd" podStartSLOduration=1.248603079 podStartE2EDuration="1.625196565s" totalImagesPullingTime="376.593486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:11:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:11:31.622677143 +0000 UTC m=+2107.119550267" watchObservedRunningTime="2026-09-16 17:11:31.625196565 +0000 UTC m=+2107.122069688"
Sep 16 17:11:33.852080 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:33.852044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:11:39.653126 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:39.653080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:11:39.754467 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:39.754429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:11:42.614119 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:11:42.614085    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bb99c5f94-rb9kd"
Sep 16 17:13:24.458443 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:13:24.458400    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:13:26.413322 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:13:26.413285    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:13:46.124727 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:13:46.124691    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:14:25.620834 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:14:25.620801    2577 scope.go:117] "RemoveContainer" containerID="11f4b096ff6d4a95817304ae6333cd0410726daf51992ee355caacbec166bc30"
Sep 16 17:14:25.629314 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:14:25.629294    2577 scope.go:117] "RemoveContainer" containerID="4816ce3aa38fd000b9ebd2666e7faa0c207cd7dead8648656c907c1853c76389"
Sep 16 17:14:37.981369 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:14:37.981327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:14:44.375933 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:14:44.375891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:14:59.544437 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:14:59.544390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:15:02.400726 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:02.400686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:15:25.652085 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:25.652001    2577 scope.go:117] "RemoveContainer" containerID="f454cdfaf03d18f116dee980b21f145d23795b9580d3b4c4d57b83518376980c"
Sep 16 17:15:25.660476 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:25.660454    2577 scope.go:117] "RemoveContainer" containerID="1c13af07615f9f91de81dc0ded28a92e7f9c39093586b8ce8696a095b4e9088e"
Sep 16 17:15:37.471547 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:37.471507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:15:50.867639 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:50.867603    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fdd987d44-5vc68_4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4/authorino/0.log"
Sep 16 17:15:56.139729 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:56.139693    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bb99c5f94-rb9kd_109a6a82-13a4-46cd-a160-a85e5f8e17ca/manager/0.log"
Sep 16 17:15:56.610978 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:56.610904    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d7c7f658-g9fdp_109611c5-d1b9-4d7f-bfa5-330861e99a0b/copy-manifests/0.log"
Sep 16 17:15:56.623334 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:56.623302    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d7c7f658-g9fdp_109611c5-d1b9-4d7f-bfa5-330861e99a0b/manager/0.log"
Sep 16 17:15:58.305561 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:58.305499    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fdd987d44-5vc68_4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4/authorino/0.log"
Sep 16 17:15:58.556697 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:58.556613    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:15:58.717486 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:58.717455    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-4jdk8_d0e098df-1e6b-4ab3-9f83-70247205fda0/kuadrant-console-plugin/0.log"
Sep 16 17:15:59.664498 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:15:59.664468    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b966d4c98-g7fbv_dd27ec48-9012-4f57-a80d-c3925b60624c/kube-auth-proxy/0.log"
Sep 16 17:16:00.008724 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:00.008629    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-7tvs6_0d1d3ce7-f283-4b11-bf5e-14ae89090257/payload-pre-processing/0.log"
Sep 16 17:16:00.241049 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:00.241017    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-695c688bd8-jc78r_493536a9-326c-41df-a8bc-cbc55a392894/router/0.log"
Sep 16 17:16:00.571207 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:00.571175    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-kr2s7_2b1c20fb-34b5-4ce7-bbb2-bf13659390aa/main/0.log"
Sep 16 17:16:00.898059 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:00.898027    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-w5n79_16b75976-64e4-4b90-960d-a9c94a85245a/main/0.log"
Sep 16 17:16:01.002467 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:01.002439    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmq9zg_62f2bda8-6656-4e2e-a890-823e9dba257e/main/0.log"
Sep 16 17:16:06.928675 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:06.928630    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mqgpr/must-gather-96v5v"]
Sep 16 17:16:06.932279 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:06.932257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqgpr/must-gather-96v5v"
Sep 16 17:16:06.934755 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:06.934736    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mqgpr\"/\"openshift-service-ca.crt\""
Sep 16 17:16:06.934873 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:06.934852    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mqgpr\"/\"kube-root-ca.crt\""
Sep 16 17:16:06.934926 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:06.934876    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mqgpr\"/\"default-dockercfg-bb76x\""
Sep 16 17:16:06.941761 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:06.941740    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqgpr/must-gather-96v5v"]
Sep 16 17:16:06.956511 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:06.956478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:16:07.054429 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.054375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log" limit=1024
Sep 16 17:16:07.102588 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.102561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g764b\" (UniqueName: \"kubernetes.io/projected/c3248ad1-d921-4289-b2fb-89ff96672770-kube-api-access-g764b\") pod \"must-gather-96v5v\" (UID: \"c3248ad1-d921-4289-b2fb-89ff96672770\") " pod="openshift-must-gather-mqgpr/must-gather-96v5v"
Sep 16 17:16:07.102774 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.102621    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c3248ad1-d921-4289-b2fb-89ff96672770-must-gather-output\") pod \"must-gather-96v5v\" (UID: \"c3248ad1-d921-4289-b2fb-89ff96672770\") " pod="openshift-must-gather-mqgpr/must-gather-96v5v"
Sep 16 17:16:07.203957 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.203882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c3248ad1-d921-4289-b2fb-89ff96672770-must-gather-output\") pod \"must-gather-96v5v\" (UID: \"c3248ad1-d921-4289-b2fb-89ff96672770\") " pod="openshift-must-gather-mqgpr/must-gather-96v5v"
Sep 16 17:16:07.204104 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.203990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g764b\" (UniqueName: \"kubernetes.io/projected/c3248ad1-d921-4289-b2fb-89ff96672770-kube-api-access-g764b\") pod \"must-gather-96v5v\" (UID: \"c3248ad1-d921-4289-b2fb-89ff96672770\") " pod="openshift-must-gather-mqgpr/must-gather-96v5v"
Sep 16 17:16:07.204241 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.204225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c3248ad1-d921-4289-b2fb-89ff96672770-must-gather-output\") pod \"must-gather-96v5v\" (UID: \"c3248ad1-d921-4289-b2fb-89ff96672770\") " pod="openshift-must-gather-mqgpr/must-gather-96v5v"
Sep 16 17:16:07.212027 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.212000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g764b\" (UniqueName: \"kubernetes.io/projected/c3248ad1-d921-4289-b2fb-89ff96672770-kube-api-access-g764b\") pod \"must-gather-96v5v\" (UID: \"c3248ad1-d921-4289-b2fb-89ff96672770\") " pod="openshift-must-gather-mqgpr/must-gather-96v5v"
Sep 16 17:16:07.241700 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.241677    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqgpr/must-gather-96v5v"
Sep 16 17:16:07.517131 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.517105    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqgpr/must-gather-96v5v"]
Sep 16 17:16:07.518568 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:16:07.518518    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3248ad1_d921_4289_b2fb_89ff96672770.slice/crio-0d804207f01512dc37cc54078f480c1c9091a7b3c1ddf8907c665b9f5df57655 WatchSource:0}: Error finding container 0d804207f01512dc37cc54078f480c1c9091a7b3c1ddf8907c665b9f5df57655: Status 404 returned error can't find the container with id 0d804207f01512dc37cc54078f480c1c9091a7b3c1ddf8907c665b9f5df57655
Sep 16 17:16:07.520460 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:07.520440    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:16:08.124085 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:08.124050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqgpr/must-gather-96v5v" event={"ID":"c3248ad1-d921-4289-b2fb-89ff96672770","Type":"ContainerStarted","Data":"0d804207f01512dc37cc54078f480c1c9091a7b3c1ddf8907c665b9f5df57655"}
Sep 16 17:16:09.137039 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:09.136999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqgpr/must-gather-96v5v" event={"ID":"c3248ad1-d921-4289-b2fb-89ff96672770","Type":"ContainerStarted","Data":"d91188d94766c1575b66101806d57f7f8ae17003b5e48df6fcbc24c0381f4920"}
Sep 16 17:16:09.137039 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:09.137044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqgpr/must-gather-96v5v" event={"ID":"c3248ad1-d921-4289-b2fb-89ff96672770","Type":"ContainerStarted","Data":"e60f43920da875f25c05e1578cfc0094fd3aadcce1d6562131c2ae7aa37a5a74"}
Sep 16 17:16:09.159100 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:09.159049    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mqgpr/must-gather-96v5v" podStartSLOduration=2.316425574 podStartE2EDuration="3.15903131s" totalImagesPullingTime="842.605736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:16:09.156838658 +0000 UTC m=+2384.653711782" watchObservedRunningTime="2026-09-16 17:16:09.15903131 +0000 UTC m=+2384.655904434"
Sep 16 17:16:10.258966 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:10.258934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-b2pqs_6d52fd17-9aa5-4d34-a16e-16220c51a87c/global-pull-secret-syncer/0.log"
Sep 16 17:16:10.371907 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:10.371870    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g4mhj_e30f3f32-2dd2-4c5c-a9a9-03328722a78f/konnectivity-agent/0.log"
Sep 16 17:16:10.534508 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:10.534416    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-51.ec2.internal_29c4f9a3e72c82af2027e396c27340f6/haproxy/0.log"
Sep 16 17:16:14.819552 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:14.818745    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fdd987d44-5vc68_4eed6d7d-7c03-4389-a6f4-9e14a1b32ac4/authorino/0.log"
Sep 16 17:16:14.914368 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:14.914336    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-4jdk8_d0e098df-1e6b-4ab3-9f83-70247205fda0/kuadrant-console-plugin/0.log"
Sep 16 17:16:16.548279 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:16.548240    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dfca1cf2-c60c-495a-9661-acc2c9a9f8de/alertmanager/0.log"
Sep 16 17:16:16.575831 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:16.575803    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dfca1cf2-c60c-495a-9661-acc2c9a9f8de/config-reloader/0.log"
Sep 16 17:16:16.600259 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:16.600222    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dfca1cf2-c60c-495a-9661-acc2c9a9f8de/kube-rbac-proxy-web/0.log"
Sep 16 17:16:16.639690 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:16.639656    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dfca1cf2-c60c-495a-9661-acc2c9a9f8de/kube-rbac-proxy/0.log"
Sep 16 17:16:16.666168 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:16.666134    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dfca1cf2-c60c-495a-9661-acc2c9a9f8de/kube-rbac-proxy-metric/0.log"
Sep 16 17:16:16.702154 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:16.702122    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dfca1cf2-c60c-495a-9661-acc2c9a9f8de/prom-label-proxy/0.log"
Sep 16 17:16:16.729505 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:16.729462    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_dfca1cf2-c60c-495a-9661-acc2c9a9f8de/init-config-reloader/0.log"
Sep 16 17:16:16.782861 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:16.782821    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-b6msc_e8989178-a442-4afa-b2ce-83cfb3677bb6/cluster-monitoring-operator/0.log"
Sep 16 17:16:17.167034 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:17.167006    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hdwzx_9d46271d-005e-4ebc-8ac4-6ac119cd0fb8/node-exporter/0.log"
Sep 16 17:16:17.198657 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:17.198626    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hdwzx_9d46271d-005e-4ebc-8ac4-6ac119cd0fb8/kube-rbac-proxy/0.log"
Sep 16 17:16:17.238097 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:17.238071    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hdwzx_9d46271d-005e-4ebc-8ac4-6ac119cd0fb8/init-textfile/0.log"
Sep 16 17:16:17.757832 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:17.757799    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55c797ccb7-8w7wg_41c6d354-0a6d-47ab-b2b8-8447825922e1/telemeter-client/0.log"
Sep 16 17:16:17.780519 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:17.780484    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55c797ccb7-8w7wg_41c6d354-0a6d-47ab-b2b8-8447825922e1/reload/0.log"
Sep 16 17:16:17.803407 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:17.803374    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55c797ccb7-8w7wg_41c6d354-0a6d-47ab-b2b8-8447825922e1/kube-rbac-proxy/0.log"
Sep 16 17:16:19.863431 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:19.863350    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:16:19.870965 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:19.870932    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/3.log"
Sep 16 17:16:19.943718 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:19.943682    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"]
Sep 16 17:16:19.950382 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:19.950356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:19.988656 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:19.988618    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"]
Sep 16 17:16:20.094994 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.094951    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-sys\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.095301 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.095029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-podres\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.095301 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.095071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tcn5\" (UniqueName: \"kubernetes.io/projected/c9a979ea-138e-49c5-874d-22b8cf7003df-kube-api-access-6tcn5\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.095301 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.095137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-proc\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.095301 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.095241    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-lib-modules\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.196088 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.196047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-lib-modules\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.196361 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.196337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-sys\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.196504 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.196490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-podres\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.196657 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.196640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tcn5\" (UniqueName: \"kubernetes.io/projected/c9a979ea-138e-49c5-874d-22b8cf7003df-kube-api-access-6tcn5\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.196778 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.196765    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-proc\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.196994 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.196977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-proc\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.197188 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.197160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-sys\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.197340 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.197229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-podres\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.197524 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.197508    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c9a979ea-138e-49c5-874d-22b8cf7003df-lib-modules\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.207852 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.207805    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tcn5\" (UniqueName: \"kubernetes.io/projected/c9a979ea-138e-49c5-874d-22b8cf7003df-kube-api-access-6tcn5\") pod \"perf-node-gather-daemonset-lz2wg\" (UID: \"c9a979ea-138e-49c5-874d-22b8cf7003df\") " pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.296139 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.296101    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:20.489987 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.489908    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cc5664dbc-8dnf5_24eeec43-4bf5-4edd-816c-4f2b351eb077/console/0.log"
Sep 16 17:16:20.596890 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:20.595304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"]
Sep 16 17:16:20.596968 ip-10-0-139-51 kubenswrapper[2577]: W0916 17:16:20.596922    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc9a979ea_138e_49c5_874d_22b8cf7003df.slice/crio-6b6f21f4486b5fc78e6f1fe08181822d298c487cb780b3856603a0f370008be8 WatchSource:0}: Error finding container 6b6f21f4486b5fc78e6f1fe08181822d298c487cb780b3856603a0f370008be8: Status 404 returned error can't find the container with id 6b6f21f4486b5fc78e6f1fe08181822d298c487cb780b3856603a0f370008be8
Sep 16 17:16:21.236547 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:21.236490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg" event={"ID":"c9a979ea-138e-49c5-874d-22b8cf7003df","Type":"ContainerStarted","Data":"0ab72e51138d45868ef9fbf975dcc7db55736ba46ce06e171fadd19dbbf4ccf4"}
Sep 16 17:16:21.237116 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:21.237088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg" event={"ID":"c9a979ea-138e-49c5-874d-22b8cf7003df","Type":"ContainerStarted","Data":"6b6f21f4486b5fc78e6f1fe08181822d298c487cb780b3856603a0f370008be8"}
Sep 16 17:16:21.238078 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:21.238057    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:21.243075 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:21.243044    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-d2cck_eced8ef1-c366-44cc-b913-f5148f7c6812/volume-data-source-validator/0.log"
Sep 16 17:16:21.256178 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:21.256125    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg" podStartSLOduration=2.256106897 podStartE2EDuration="2.256106897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:16:21.253877255 +0000 UTC m=+2396.750750380" watchObservedRunningTime="2026-09-16 17:16:21.256106897 +0000 UTC m=+2396.752980022"
Sep 16 17:16:22.426956 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:22.426916    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b5bfg_d512fb78-3c62-4f8c-af0a-3eb4a888d60d/dns/0.log"
Sep 16 17:16:22.448569 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:22.448538    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b5bfg_d512fb78-3c62-4f8c-af0a-3eb4a888d60d/kube-rbac-proxy/0.log"
Sep 16 17:16:22.546372 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:22.546346    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-htltb_83be222b-05e2-44b0-90c5-bf387e175d3e/dns-node-resolver/0.log"
Sep 16 17:16:23.098099 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:23.098057    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67d8774cb9-gwd48_2d0d8200-94cf-41bb-8985-9cb2d04dea16/registry/0.log"
Sep 16 17:16:23.121827 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:23.121799    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ft9d_b1d7e520-75a5-4587-b4f3-77ecdf1b2460/node-ca/0.log"
Sep 16 17:16:24.452885 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:24.452855    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b966d4c98-g7fbv_dd27ec48-9012-4f57-a80d-c3925b60624c/kube-auth-proxy/0.log"
Sep 16 17:16:24.565041 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:24.565011    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-7tvs6_0d1d3ce7-f283-4b11-bf5e-14ae89090257/payload-pre-processing/0.log"
Sep 16 17:16:24.621612 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:24.621577    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-695c688bd8-jc78r_493536a9-326c-41df-a8bc-cbc55a392894/router/0.log"
Sep 16 17:16:25.139729 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:25.139697    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:16:25.152978 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:25.152297    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8trxf_911d89a7-998b-43e4-a787-8c3f14775db3/console-operator/2.log"
Sep 16 17:16:25.545586 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:25.545478    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rds89_4423a1a4-565b-402b-a813-b228a1d2213c/serve-healthcheck-canary/0.log"
Sep 16 17:16:25.685371 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:25.685337    2577 scope.go:117] "RemoveContainer" containerID="f4e9a3958592af37143a0299e6d95b5ab608ba30855bb50ba2fabaa82b9ce5bd"
Sep 16 17:16:26.071694 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:26.071665    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lb6ps_a0fcb09c-502d-4494-99fc-a128ca504f2e/kube-rbac-proxy/0.log"
Sep 16 17:16:26.095489 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:26.095459    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lb6ps_a0fcb09c-502d-4494-99fc-a128ca504f2e/exporter/0.log"
Sep 16 17:16:26.120694 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:26.120663    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lb6ps_a0fcb09c-502d-4494-99fc-a128ca504f2e/extractor/0.log"
Sep 16 17:16:28.263066 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:28.263041    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mqgpr/perf-node-gather-daemonset-lz2wg"
Sep 16 17:16:28.692744 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:28.692698    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6bb99c5f94-rb9kd_109a6a82-13a4-46cd-a160-a85e5f8e17ca/manager/0.log"
Sep 16 17:16:28.930120 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:28.930085    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d7c7f658-g9fdp_109611c5-d1b9-4d7f-bfa5-330861e99a0b/manager/0.log"
Sep 16 17:16:28.953216 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:28.953156    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54d7c7f658-g9fdp_109611c5-d1b9-4d7f-bfa5-330861e99a0b/copy-manifests/0.log"
Sep 16 17:16:30.432241 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:30.432208    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-c5d8fb6c6-gh4q6_b84afc05-e3a7-4031-9277-1ed5e9b6a7dc/manager/0.log"
Sep 16 17:16:30.504961 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:30.504796    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-r2clv_9ef8e48a-7574-4be7-9c6f-18addad37c36/openshift-lws-operator/0.log"
Sep 16 17:16:30.771436 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:30.771357    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b966d4c98-g7fbv_dd27ec48-9012-4f57-a80d-c3925b60624c/kube-auth-proxy/0.log"
Sep 16 17:16:30.850167 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:30.850127    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-7tvs6_0d1d3ce7-f283-4b11-bf5e-14ae89090257/payload-pre-processing/0.log"
Sep 16 17:16:30.897359 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:30.897332    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-695c688bd8-jc78r_493536a9-326c-41df-a8bc-cbc55a392894/router/0.log"
Sep 16 17:16:32.815231 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:32.815197    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b966d4c98-g7fbv_dd27ec48-9012-4f57-a80d-c3925b60624c/kube-auth-proxy/0.log"
Sep 16 17:16:32.889095 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:32.889056    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-7tvs6_0d1d3ce7-f283-4b11-bf5e-14ae89090257/payload-pre-processing/0.log"
Sep 16 17:16:32.940325 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:32.940245    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-695c688bd8-jc78r_493536a9-326c-41df-a8bc-cbc55a392894/router/0.log"
Sep 16 17:16:35.975362 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:35.975326    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-jxzb7_5d100c30-471d-45a7-ae6e-4c4610cfb308/kube-storage-version-migrator-operator/1.log"
Sep 16 17:16:35.977344 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:35.977313    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-jxzb7_5d100c30-471d-45a7-ae6e-4c4610cfb308/kube-storage-version-migrator-operator/0.log"
Sep 16 17:16:37.219648 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.219621    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p7zhj_d2570fd8-d0e6-4ce4-a62c-65e9b5c22533/kube-multus-additional-cni-plugins/0.log"
Sep 16 17:16:37.241191 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.241164    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p7zhj_d2570fd8-d0e6-4ce4-a62c-65e9b5c22533/egress-router-binary-copy/0.log"
Sep 16 17:16:37.261078 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.261055    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p7zhj_d2570fd8-d0e6-4ce4-a62c-65e9b5c22533/cni-plugins/0.log"
Sep 16 17:16:37.280997 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.280975    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p7zhj_d2570fd8-d0e6-4ce4-a62c-65e9b5c22533/bond-cni-plugin/0.log"
Sep 16 17:16:37.301885 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.301861    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p7zhj_d2570fd8-d0e6-4ce4-a62c-65e9b5c22533/routeoverride-cni/0.log"
Sep 16 17:16:37.322575 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.322549    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p7zhj_d2570fd8-d0e6-4ce4-a62c-65e9b5c22533/whereabouts-cni-bincopy/0.log"
Sep 16 17:16:37.343382 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.343359    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p7zhj_d2570fd8-d0e6-4ce4-a62c-65e9b5c22533/whereabouts-cni/0.log"
Sep 16 17:16:37.423407 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.423350    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fpwb6_6ebfc096-7519-4787-a95a-b39896f1123d/kube-multus/0.log"
Sep 16 17:16:37.536117 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.536045    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9wkss_07fb4aaa-bae1-4483-8001-e6bbd85f02e4/network-metrics-daemon/0.log"
Sep 16 17:16:37.553315 ip-10-0-139-51 kubenswrapper[2577]: I0916 17:16:37.553291    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9wkss_07fb4aaa-bae1-4483-8001-e6bbd85f02e4/kube-rbac-proxy/0.log"